Skip to main content

Better tracing of Npgsql connection pool usage in the works



Sometimes, Npgsql users receive errors when working with connection pooling. The problem appears when they try to open a new connection and receive the following message: 

"Timeout while getting a connection from pool."

This is caused when an attempt is made to get a connection from a pool which  has all its connections used. Most of the time this problem is a difficult one to track because it generally happens when the system is in production and not in the development phase. Of course. :)

Some time ago, I received a report from Miłosz Kubański about such a problem. I told him I would work in a way to get more information so we could check what was happening.

In order to help us find the problem, I added a little "hack" to Npgsql: Whenever an error while getting a connection from the pool occurs, Npgsql would log a stacktrace of the allocation of all the connections which were in the pool. Theoretically, those connections which were allocated should not be retained and by taking a "snapshot" of those allocations, we could get some tips about which code allocated the connection and check for potential missing releases.

Last week I sent Milosz this modified version of Npgsql so he could give it a try and send me back the log. Yesterday he sent me the log. Inside it, I chose two stacktraces and asked him if would be possible there was any missing close.

Milosz replied saying there was indeed a missing close call. It was an exception inside a very complex system. 

Based on this feedback, I think this "hack" has shown to be very useful in future cases and I'll work to get this change in a "non-hack" status and add it to Npgsql code. I hope this helps Npgsql users to find possible causes for connection pooling problems.

When this feature is available I'll let you know.



Comments

Unknown said…
This would be very useful -- any timeframe on when this feature will be available?
Rui Guimarães said…
This seams really useful. Do you have any update on this?
Unknown said…
Francisco, did the "hack" ever made it to Npgsql 2.1.0 or 2.1.1?
Hi all!

Sorry, but this feature isn't present in current Npgsql release. I'm working to add it to master branch and I hope it will be present in the 2.2 release.
Unknown said…
Hi, This 'Hack' would be useful for an issue we are investigating right now. Is this logging available in the 2.2 beta 1 release?

Thanks.
Hi, all!

We have an issue open on github about that:
https://github.com/npgsql/Npgsql/issues/237

I already have a patch to include this feature. Unfortunately, I think it will be present in the 2.3 release as the 2.2 is already in the RC phase.

Please, check the 237 issue above for news about this feature. I'll update the patch so it can be cleanly applied and you all can test it. This would be very helpful.

Thanks in advance and sorry for not having this feature released yet.
I just added more information to the issue tracker. I also updated the patch and created binaries with this patch applied which you can give it a try.
Check out https://github.com/npgsql/Npgsql/pull/311 for more info.

I hope it helps.

Popular posts from this blog

Npgsql Tips: Using " in (...)" queries with parameters list and "any" operator

Hi, all! We have received some users questions about how to send a list of values to be used in queries using the "in" operator. Something like: select foo, bar from table where foo in (blah1, blah2, blah3); Npgsql supports array-like parameter values and the first idea to have this working would try to use it directly: NpgsqlCommand command = new NpgsqlCommand("select * from tablee where field_serial in (:parameterlist)", conn); ArrayList l = new ArrayList(); l.Add(5); l.Add(6); command.Parameters.Add(new NpgsqlParameter("parameterlist", NpgsqlDbType.Array | NpgsqlDbType.Integer)); command.Parameters[0].Value = l.ToArray(); NpgsqlDataReader dr = command.ExecuteReader(); but unfortunately this won't work as expected. Npgsql will send a query like this: select * from tablee where field_serial in ((array[5,6])::int4[]) And Postgresql will complain with the followin...

Using Entity Framework 6 with Npgsql 2.1.0

UPDATE (2014-05-19): Marek Beneš noticed a problem in the default connection factory config. It is fixed now. Thanks, Marek! UPDATE (2014-02-20): I created a new post explaining how to get Npgsql 2.1.0. Although this post is about EF 6, I'd like to talk about our current situation to support both EF 6 and EF4.x which explain why there are some subtle changes between EF 4.x and EF 6.x App.config settings.  Support for EF versions 4.x and 6.x Sometime after we started to work on Npgsql 2.1.0, we started to add code to support EF6 and decided to reorganize our Entity Framework support code. Shay created a pull request to organize this change and isolate the EF code out of core Npgsql code. The result was the creation of two separated assemblies: Npgsql.EntityFramework.dll for EF6 and above; Npgsql.EntityFrameworkLegacy.dll for EF4.x. Only when using Npgsql with EF6 you will need to reference Npgsql.EntityFramework.dll assembly. This is needed because the EF ...

Npgsql Design time support preview available for download!

UPDATE2 : For some reason, Blogger lost the original post. I had to republish it. UPDATE : Jerónimo told me that you  must  use the Npgsql version which is inside the zip file in order to make it work. After so much time (more than I wanted it to take) Npgsql finally has a initial design time support for Visual Studio! This work was done by Jerónimo Milea. Jerónimo let me know he was working on DDEX support on this thread  after I said I was working on DDEX support for Npgsql. He sent me his working copy and I started to play with it. Note that as a preview version many things may not work ok and we wanted you to provide us feedback so we can fix any bugs. You can download the project file from our downloads page . Design time support is provided as a zip file containing support code as well as a copy of Npgsql project file. So, everything you need to start working with design time support is already packaged for you. When you unzip the file, you will have ...