Skip to main content

Npgsql Connection Pool Explained

Hi all!

From time to time, we receive some questions regarding connection pool in Npgsql and I think I should post some info about its current design.

Npgsql connection pool implements the common pattern of having some connections open beforehand so when one is needed, it will be readily available for using.

How it works

When a application opens a connection, Npgsql tries to find a pool of connections based on the connection string. If a pool doesn't exist, it is created with a number of connections specified in the MinPoolSize connectionstring parameter. After that, a connection is retrieved from this pool.

The min and max number of connections created in each pool is controlled by connection string parameters called MinPoolSize and MaxPoolSize respectively. This way, users can fine tune the pool behavior to match their scalability needs.

Npgsql controls the lifetime of unused connections in the pool, trying to get connections number near the minimum value set by user. This is done by closing unused connections which are open far long than NpgsqlConnection.ConnectionLifeTime. This control is helpful in a scenario where application uses a lot of connections in a peak situation and later goes back to normal connection usage. Those "extra" connections will stay open but won't be used anytime soon, so instead of laying there consuming server resources, Npgsql simply closes them when their lifetime is reached.

Applications also can clear a pool or all pools by using NpgsqlConnection.ClearPool() and NpgsqlConnection.ClearAllPools() static methods.


EOF Error Message

There is one error message which appears in server log with applications which use Npgsql with pooled connections. This is the error message:


LOG: unexpected EOF on client connection

This is generally caused when the application is terminated and there are connections in the pool. The Tcp connection is closed by the .Net framework without Npgsql sending the Terminate message. Sending the Terminate message to all open connections would be the best thing to do, but Npgsql, by itself, isn't be able to know when the application is being terminated and so the log is generated. According to docs, this disconnection will make the backend clean up and terminate the connection ok. So, the only drawback of this situation is this message log.

In order to get more information about Npgsql connection pooling, you may check the NpgsqlConnectorPool.cs file.

I hope this information helps developers to understand better how connection pool works with Npgsql.

If you have any other questions, please drop by Npgsql forums.

Comments

Sergio said…
This comment has been removed by the author.

Popular posts from this blog

Npgsql Tips: Using " in (...)" queries with parameters list and "any" operator

Hi, all! We have received some users questions about how to send a list of values to be used in queries using the "in" operator. Something like: select foo, bar from table where foo in (blah1, blah2, blah3); Npgsql supports array-like parameter values and the first idea to have this working would try to use it directly: NpgsqlCommand command = new NpgsqlCommand("select * from tablee where field_serial in (:parameterlist)", conn); ArrayList l = new ArrayList(); l.Add(5); l.Add(6); command.Parameters.Add(new NpgsqlParameter("parameterlist", NpgsqlDbType.Array | NpgsqlDbType.Integer)); command.Parameters[0].Value = l.ToArray(); NpgsqlDataReader dr = command.ExecuteReader(); but unfortunately this won't work as expected. Npgsql will send a query like this: select * from tablee where field_serial in ((array[5,6])::int4[]) And Postgresql will complain with the followin...

Fixed! LOG: unexpected EOF on client connection

Hi all! Since we implemented connection pool in Npgsql, we received some complaints about EOF log messages being generated on Postgresql logs when using Npgsql. This was caused by Npgsql not sending the proper terminate message to Postgresql on pooled connections when the application terminated or more specifically when the assembly was unloaded. This is a long time problem with Npgsql connection pool. I even talked about it in the past . Up to now, I had no idea about how to fix that as I wasn't able to close the connections in the pool. When I tried to put a finalizer in NpgsqlConnectorPool, which would be triggered when the assembly was unloaded, I received object already disposed exceptions when trying to send something to the stream. That's when I came up with the "excellent" idea of subclassing the networkstream class and override its Dispose method so that I could send the postgresql terminate message before it was disposed! :) It worked like a charm! ...

Npgsql 2.2.0 final release is out!

This is Npgsql 2.2.0 Final Release This release contains 249 commits since the last stable release. Includes bug fixes, improvements and new features. Update notice: If you have been using Npgsql 2.2.0-rc2, you don't need to update to this version. They are the same except for the Assembly version information. Major highlights Visual Studio DDEX support   Kenji Uno added support for DDEX. Now you can use Npgsql with Visual Studio data designer. This is a missing feature a lot of our users requested in the past. Kenji added a tutorial about how to use Npgsql with DDEX. You can find it here: https://github.com/npgsql/Npgsql/wiki/Visual-Studio-Design-Time-Support---DDEX-Provider#install-npgsqlddexprovidervsix   Entity Framework   David Karlaš added support for EFMigration and Database creation in EF6+. Now it is possible to start Code First projects without needing to create a database upfront. EntityFramework and Npgsql will take care of it. E...