Skip to main content

Better tracing of Npgsql connection pool usage in the works



Sometimes, Npgsql users receive errors when working with connection pooling. The problem appears when they try to open a new connection and receive the following message: 

"Timeout while getting a connection from pool."

This is caused when an attempt is made to get a connection from a pool which  has all its connections used. Most of the time this problem is a difficult one to track because it generally happens when the system is in production and not in the development phase. Of course. :)

Some time ago, I received a report from Miłosz Kubański about such a problem. I told him I would work in a way to get more information so we could check what was happening.

In order to help us find the problem, I added a little "hack" to Npgsql: Whenever an error while getting a connection from the pool occurs, Npgsql would log a stacktrace of the allocation of all the connections which were in the pool. Theoretically, those connections which were allocated should not be retained and by taking a "snapshot" of those allocations, we could get some tips about which code allocated the connection and check for potential missing releases.

Last week I sent Milosz this modified version of Npgsql so he could give it a try and send me back the log. Yesterday he sent me the log. Inside it, I chose two stacktraces and asked him if would be possible there was any missing close.

Milosz replied saying there was indeed a missing close call. It was an exception inside a very complex system. 

Based on this feedback, I think this "hack" has shown to be very useful in future cases and I'll work to get this change in a "non-hack" status and add it to Npgsql code. I hope this helps Npgsql users to find possible causes for connection pooling problems.

When this feature is available I'll let you know.



Comments

Unknown said…
This would be very useful -- any timeframe on when this feature will be available?
Rui Guimarães said…
This seams really useful. Do you have any update on this?
Unknown said…
Francisco, did the "hack" ever made it to Npgsql 2.1.0 or 2.1.1?
Hi all!

Sorry, but this feature isn't present in current Npgsql release. I'm working to add it to master branch and I hope it will be present in the 2.2 release.
Unknown said…
Hi, This 'Hack' would be useful for an issue we are investigating right now. Is this logging available in the 2.2 beta 1 release?

Thanks.
Hi, all!

We have an issue open on github about that:
https://github.com/npgsql/Npgsql/issues/237

I already have a patch to include this feature. Unfortunately, I think it will be present in the 2.3 release as the 2.2 is already in the RC phase.

Please, check the 237 issue above for news about this feature. I'll update the patch so it can be cleanly applied and you all can test it. This would be very helpful.

Thanks in advance and sorry for not having this feature released yet.
I just added more information to the issue tracker. I also updated the patch and created binaries with this patch applied which you can give it a try.
Check out https://github.com/npgsql/Npgsql/pull/311 for more info.

I hope it helps.

Popular posts from this blog

UUID datatype and COPY IN/OUT support added to cvs

Hi all! It was just added support to uuid datatype in cvs head. This type will be available in next Postgresql release 8.3. Thanks to David Bachmann for his patch! You can get more info about this patch in this mailing list post . Also was added support for copy in and copy out operations. Now, users can provide streams which can be copied directly to and from Postgresql tables! Thanks to Kalle Hallivuori for providing a patch! Thanks to Truviso for giving support to Kalle. More info about that including a demo and ready to use compiled Npgsql.dll versions can be found here . That's it! As soon as we get more features added, I will post info about them here. Stay tuned! :)

Npgsql Tips: Using " in (...)" queries with parameters list and "any" operator

Hi, all! We have received some users questions about how to send a list of values to be used in queries using the "in" operator. Something like: select foo, bar from table where foo in (blah1, blah2, blah3); Npgsql supports array-like parameter values and the first idea to have this working would try to use it directly: NpgsqlCommand command = new NpgsqlCommand("select * from tablee where field_serial in (:parameterlist)", conn); ArrayList l = new ArrayList(); l.Add(5); l.Add(6); command.Parameters.Add(new NpgsqlParameter("parameterlist", NpgsqlDbType.Array | NpgsqlDbType.Integer)); command.Parameters[0].Value = l.ToArray(); NpgsqlDataReader dr = command.ExecuteReader(); but unfortunately this won't work as expected. Npgsql will send a query like this: select * from tablee where field_serial in ((array[5,6])::int4[]) And Postgresql will complain with the followin...

Using Entity Framework 6 with Npgsql 2.1.0

UPDATE (2014-05-19): Marek Beneš noticed a problem in the default connection factory config. It is fixed now. Thanks, Marek! UPDATE (2014-02-20): I created a new post explaining how to get Npgsql 2.1.0. Although this post is about EF 6, I'd like to talk about our current situation to support both EF 6 and EF4.x which explain why there are some subtle changes between EF 4.x and EF 6.x App.config settings.  Support for EF versions 4.x and 6.x Sometime after we started to work on Npgsql 2.1.0, we started to add code to support EF6 and decided to reorganize our Entity Framework support code. Shay created a pull request to organize this change and isolate the EF code out of core Npgsql code. The result was the creation of two separated assemblies: Npgsql.EntityFramework.dll for EF6 and above; Npgsql.EntityFrameworkLegacy.dll for EF4.x. Only when using Npgsql with EF6 you will need to reference Npgsql.EntityFramework.dll assembly. This is needed because the EF ...