Why is storage usage increasing consistently and overloading our Azure Postgresql server?

Question

Why is storage usage increasing consistently and overloading our Azure Postgresql server?

Khan, Sifat Shahriar 5

Storage usage has been consistently increasing in our Postgresql server since May 28th until yesterday, when it got overloaded. The effective size of the database is around 20GB, while the database storage used has reached 120+ GB. Why is it happening?
Please note that the database size, meaning all our tables combined, is around 20 GB. Per day, we ingest around 200MB of data.
We switched from Burstable Compute to General Compute on May 28th, where we ran into some issue, where our server had crashed and needed to be restarted from the backend with the help of Azure-tech-support.

User's image

Khan, Sifat Shahriar 5 Reputation points

2025-06-16T20:56:01.2966667+00:00

Transaction Log Storage might be the culprit?
Khan, Sifat Shahriar 5 Reputation points

2025-06-16T21:12:44.6733333+00:00

On a separate note, why does Microsoft have to gray out the NEXT button. If you don't want to provide me the service of letting me create a support request directly and want me to wait here on the Q&A platform for assistance, that's fine, just add that explanation in the page and remove the NEXT button. Because, people end up wasting time trying to figure out, how to make the support request, and eventually giving up. Instead, have a clear communication, or even show much extra it would cost me to avail that service.
Sai Raghunadh M 4,210 Reputation points Microsoft External Staff Moderator

2025-06-16T23:07:19.2633333+00:00

Hi @ Khan, Sifat Shahriar

We understand you have questions about the Developer support plan.

Microsoft Q&A is our first party question and answer platform where enthusiasts, MVPs, and other community experts can engage. The developer support plan that you have purchased also ensures a new experience where you receive business hours access to Support Engineers on Microsoft Q&A with prioritized responses, as the first line of support.

Additionally, if you have issues pertaining to Azure billing or subscription management, you can always create a support request though the Azure Portal or by following this link.

We would be more than happy to assist you with any technical questions you have here on Q&A. Any time you post a new thread; you will receive a response within 8 business hours (see here for more details on the SLA and subscription benefits).

Please refer to this Priority Community Support FAQs for Azure Developer Support Plan: What if my issue requires a support ticket?"
Sai Raghunadh M 4,210 Reputation points Microsoft External Staff Moderator

2025-06-16T23:13:37.7233333+00:00

Thank you for reaching out regarding the unexpected storage increase in your Azure PostgreSQL server. Based on your description, the storage growth could be due to several factors beyond the actual database size, such as:

Transaction Logs (WAL Files) – If WAL files are not being archived or cleaned up, they may consume excessive storage.

Dead Tuples & Vacuuming Issues – Inefficient autovacuum settings can lead to accumulation of outdated rows.

Table & Index Bloat – Frequent updates and deletes can increase storage usage.

Temporary Files & Debug Logs – Large queries or server logs might be occupying additional space.

Compute Configuration Changes – The switch from Burstable to General Compute may have impacted PostgreSQL’s storage behavior.

Please refer the below Microsoft Documentation that might help you: https://techcommunity.microsoft.com/blog/azuredbsupport/learning-from-expertise-7-where-is-my-server-storage-taken---azure-postgresql/3032964

Hope this helps. Do let us know if you any further queries.
Khan, Sifat Shahriar 5 Reputation points

2025-06-17T14:00:04.6966667+00:00

27GB of temp file
Khan, Sifat Shahriar 5 Reputation points

2025-06-17T14:09:00.5966667+00:00

Are there any settings that I am missing, that auto-cleans these temp files?
Sai Raghunadh M 4,210 Reputation points Microsoft External Staff Moderator

2025-06-17T19:45:55.14+00:00
Hi @ Khan, Sifat Shahriar

To clarify, in Azure Database for PostgreSQL, temporary files are typically cleaned up automatically when a query or session ends. However, if you're seeing large or persistent temp files (like the 27GB you've found), it may indicate:

Long-running or idle queries still holding temp space

Queries exceeding work_mem, causing frequent disk spills

Orphaned files from previous server crashes or restarts

While there isn’t a direct setting in the managed Azure PostgreSQL environment to auto-clean these temp files, here are some practical steps you can take to mitigate and monitor them:

1**)** Use pg_stat_activity to identify long-running queries that might be holding onto temp space.

Run the following to see which databases are consuming temp files:

SELECT datname, temp_files, temp_bytes/1024/1024 AS temp_mb FROM pg_stat_database ORDER BY temp_mb DESC;

If work_mem is too low, even moderately sized queries may spill to disk. Consider gradually increasing it based on workload.

If temp files are leftover from past crashes (as may have happened after the May 28 incident), a restart can sometimes clear them out.

Please go through the similar discussion on Temp files that might help you:

https://learn.microsoft.com/en-us/answers/questions/2069089/disk-space-error-and-temp-files-management-in-azur

1 answer

Your answer

Khan, Sifat Shahriar 5 Reputation points

2025-06-16T20:56:01.2966667+00:00

Transaction Log Storage might be the culprit?
Khan, Sifat Shahriar 5 Reputation points

2025-06-16T21:12:44.6733333+00:00

On a separate note, why does Microsoft have to gray out the NEXT button. If you don't want to provide me the service of letting me create a support request directly and want me to wait here on the Q&A platform for assistance, that's fine, just add that explanation in the page and remove the NEXT button. Because, people end up wasting time trying to figure out, how to make the support request, and eventually giving up. Instead, have a clear communication, or even show much extra it would cost me to avail that service.
Sai Raghunadh M 4,210 Reputation points Microsoft External Staff Moderator

2025-06-16T23:07:19.2633333+00:00

Hi @ Khan, Sifat Shahriar

We understand you have questions about the Developer support plan.

Microsoft Q&A is our first party question and answer platform where enthusiasts, MVPs, and other community experts can engage. The developer support plan that you have purchased also ensures a new experience where you receive business hours access to Support Engineers on Microsoft Q&A with prioritized responses, as the first line of support.

Additionally, if you have issues pertaining to Azure billing or subscription management, you can always create a support request though the Azure Portal or by following this link.

We would be more than happy to assist you with any technical questions you have here on Q&A. Any time you post a new thread; you will receive a response within 8 business hours (see here for more details on the SLA and subscription benefits).

Please refer to this Priority Community Support FAQs for Azure Developer Support Plan: What if my issue requires a support ticket?"
Sai Raghunadh M 4,210 Reputation points Microsoft External Staff Moderator

2025-06-16T23:13:37.7233333+00:00

Thank you for reaching out regarding the unexpected storage increase in your Azure PostgreSQL server. Based on your description, the storage growth could be due to several factors beyond the actual database size, such as:

Transaction Logs (WAL Files) – If WAL files are not being archived or cleaned up, they may consume excessive storage.

Dead Tuples & Vacuuming Issues – Inefficient autovacuum settings can lead to accumulation of outdated rows.

Table & Index Bloat – Frequent updates and deletes can increase storage usage.

Temporary Files & Debug Logs – Large queries or server logs might be occupying additional space.

Compute Configuration Changes – The switch from Burstable to General Compute may have impacted PostgreSQL’s storage behavior.

Please refer the below Microsoft Documentation that might help you: https://techcommunity.microsoft.com/blog/azuredbsupport/learning-from-expertise-7-where-is-my-server-storage-taken---azure-postgresql/3032964

Hope this helps. Do let us know if you any further queries.
Khan, Sifat Shahriar 5 Reputation points

2025-06-17T14:00:04.6966667+00:00

27GB of temp file
Khan, Sifat Shahriar 5 Reputation points

2025-06-17T14:09:00.5966667+00:00

Are there any settings that I am missing, that auto-cleans these temp files?
Sai Raghunadh M 4,210 Reputation points Microsoft External Staff Moderator

2025-06-17T19:45:55.14+00:00

Hi @ Khan, Sifat Shahriar

To clarify, in Azure Database for PostgreSQL, temporary files are typically cleaned up automatically when a query or session ends. However, if you're seeing large or persistent temp files (like the 27GB you've found), it may indicate:

Long-running or idle queries still holding temp space

Queries exceeding work_mem, causing frequent disk spills

Orphaned files from previous server crashes or restarts

While there isn’t a direct setting in the managed Azure PostgreSQL environment to auto-clean these temp files, here are some practical steps you can take to mitigate and monitor them:

1**)** Use pg_stat_activity to identify long-running queries that might be holding onto temp space.

Run the following to see which databases are consuming temp files:

SELECT datname, temp_files, temp_bytes/1024/1024 AS temp_mb FROM pg_stat_database ORDER BY temp_mb DESC;

If work_mem is too low, even moderately sized queries may spill to disk. Consider gradually increasing it based on workload.

If temp files are leftover from past crashes (as may have happened after the May 28 incident), a restart can sometimes clear them out.

Please go through the similar discussion on Temp files that might help you:

https://learn.microsoft.com/en-us/answers/questions/2069089/disk-space-error-and-temp-files-management-in-azur

Answer 1

Quick Update on the Database Issue ( Resolved)

One of my team members connected with Microsoft Technical Support and successfully resolved the problem.

What Happened:

When we switched the database tier from Burstable to General Purpose on May 28th, the database was under heavy load (CPU usage at 100%). Because of that, the SAAS token — an internal authentication token used by Microsoft for PostgreSQL — wasn’t refreshed properly.

Since the token wasn’t refreshed, the system started saving WAL (Write-Ahead Logging) files in the background. These files are usually managed automatically, but the failure in token refresh prevented that process. As a result, around 100GB of WAL files accumulated between May 19th and June 15th.

Solution:

Microsoft’s support team:

Cleared all the accumulated WAL files
Manually refreshed the SAAS token

This is not something we have access to manage on our side — only Microsoft’s authorized personnel can perform these actions.

Current Status:

The database is now stable and performing well. Everything looks good on the backend too.

Sai Raghunadh M 4,210 Reputation points Microsoft External Staff Moderator

2025-06-17T21:21:41.38+00:00

@ Khan, Sifat Shahriar

Glad to know that your issue got resolved. Please continue using Microsoft Q&A.

Share via

Why is storage usage increasing consistently and overloading our Azure Postgresql server?

1 answer

Your answer