Tue, 31 Mar 2020 18:22:05 +0200 [server] Make connection pooler configurable and set better default values draft default tip
Philippe Pepiot <ph@itsalwaysdns.eu> [Tue, 31 Mar 2020 18:22:05 +0200] rev 12966
[server] Make connection pooler configurable and set better default values Drop the configuration connections-pool-size and add new configurations options: * connections-pool-min-size. Set to 0 by default so we open connections only when needed. This avoid opening min-size*processes connections at startup, which is, it think, a good default. * connections-pool-max-size. Set to 0 (unlimited) by default, so we move the bottleneck to postgresql. * connections-idle-timeout. Set to 10 minutes. I don't have arguments about this except that this is the default in pgbouncer.
Tue, 31 Mar 2020 18:12:20 +0200 [server] Enhance connections-pooler-enabled documentation draft
Philippe Pepiot <ph@itsalwaysdns.eu> [Tue, 31 Mar 2020 18:12:20 +0200] rev 12965
[server] Enhance connections-pooler-enabled documentation
Tue, 31 Mar 2020 16:17:14 +0200 [server] move connection pooler initialization logic to get_cnxset() draft
Philippe Pepiot <ph@itsalwaysdns.eu> [Tue, 31 Mar 2020 16:17:14 +0200] rev 12964
[server] move connection pooler initialization logic to get_cnxset() This avoid complex logic in Repository initialization.
Mon, 30 Mar 2020 15:46:12 +0200 [server] dynamically close idle database connections draft
Philippe Pepiot <ph@itsalwaysdns.eu> [Mon, 30 Mar 2020 15:46:12 +0200] rev 12963
[server] dynamically close idle database connections When pool hasn't been empty for `idle_timeout` time, start closing connections.
Mon, 30 Mar 2020 15:45:40 +0200 [server] implement dynamic database pooler draft
Philippe Pepiot <ph@itsalwaysdns.eu> [Mon, 30 Mar 2020 15:45:40 +0200] rev 12962
[server] implement dynamic database pooler Opening too much database connection has a cost at startup and also PostgreSQL as a maximum number of connection (100 by default). This get worse when starting multiple wsgi processes, since each process has its own database pool. Instead of opening `connections-pool-size` connections to the database at startup, just open one and open more only when needed.
Mon, 30 Mar 2020 15:30:02 +0200 [server] use a LifoQueue in _CnxSetPool draft
Philippe Pepiot <ph@itsalwaysdns.eu> [Mon, 30 Mar 2020 15:30:02 +0200] rev 12961
[server] use a LifoQueue in _CnxSetPool In postgresql, some cache is attached to the connection. Using a LifoQueue (last-in, first-out) makes a few connections to get the most load which give best performance.
Mon, 30 Mar 2020 15:23:56 +0200 [server] extract creating a new cnxset in a _new_cnxset() helper draft
Philippe Pepiot <ph@itsalwaysdns.eu> [Mon, 30 Mar 2020 15:23:56 +0200] rev 12960
[server] extract creating a new cnxset in a _new_cnxset() helper So we can move logic specific to _CnxSetPool here.
Mon, 30 Mar 2020 15:19:23 +0200 [server] avoid a possible race condition on _CnxSetPool.close() draft
Philippe Pepiot <ph@itsalwaysdns.eu> [Mon, 30 Mar 2020 15:19:23 +0200] rev 12959
[server] avoid a possible race condition on _CnxSetPool.close() The pool could become empty between time to check and time to use.
Mon, 30 Mar 2020 15:17:10 +0200 [server] extract "no pooler" CnxSet class to a _BaseCnxSet class draft
Philippe Pepiot <ph@itsalwaysdns.eu> [Mon, 30 Mar 2020 15:17:10 +0200] rev 12958
[server] extract "no pooler" CnxSet class to a _BaseCnxSet class So we get rid of "if self._queue is None" in each method of _CnxSetPool Also add helper get_cnxset(source, size) to instantiate the correct pooler class.
Tue, 31 Mar 2020 19:15:03 +0200 [server] prevent returning closed cursor to the database pool draft
Philippe Pepiot <ph@itsalwaysdns.eu> [Tue, 31 Mar 2020 19:15:03 +0200] rev 12957
[server] prevent returning closed cursor to the database pool In since c8c6ad8 init_repository use repo.internal_cnx() instead of repo.system_source.get_connection() so it use the pool and we should not close cursors from the pool before returning it back. Otherwise we may have "connection already closed" error. This bug only trigger when connection-pool-size = 1. Since we are moving to use a dynamic pooler we need to get this fixed. This does not occur with sqlite since the connection wrapper instantiate new cursor everytime, but this occur with other databases.
Fri, 20 Mar 2020 14:34:07 +0100 [rql] Store selected variables for RQL select queries in ResultSet (#17218476)
Laurent Wouters <lwouters@cenotelie.fr> [Fri, 20 Mar 2020 14:34:07 +0100] rev 12931
[rql] Store selected variables for RQL select queries in ResultSet (#17218476) By storing the name of the selected variables for RQL select queries in the ResultSet (within the "variables" attribute), the information can be passed down to specific protocols, e.g. rqlio that may wish to pass is down further to clients. In turn, clients can then choose to present the results of RQL select queries as symbolic bindings using the names used in the query's projection, instead of ordinal arrays.
Thu, 12 Mar 2020 15:38:51 +0100 [debugtoolbar] Make SQL and RQL tables sortable
Simon Chabot <simon.chabot@logilab.fr> [Thu, 12 Mar 2020 15:38:51 +0100] rev 12930
[debugtoolbar] Make SQL and RQL tables sortable It's usefull to sort query by duration for instance.
Thu, 12 Mar 2020 15:37:59 +0100 [debugtoolbar] Do not render CW controller panel when no controller got collected
Simon Chabot <simon.chabot@logilab.fr> [Thu, 12 Mar 2020 15:37:59 +0100] rev 12929
[debugtoolbar] Do not render CW controller panel when no controller got collected This can happen on pure pyramid applications for instance.
Wed, 18 Mar 2020 11:30:05 +0100 [doc] Fix bugs in code examples
Elodie Thieblin <ethieblin@logilab.fr> [Wed, 18 Mar 2020 11:30:05 +0100] rev 12928
[doc] Fix bugs in code examples
Wed, 18 Mar 2020 11:29:30 +0100 [doc] Remove zone cube which causes to crash on install
Elodie Thieblin <ethieblin@logilab.fr> [Wed, 18 Mar 2020 11:29:30 +0100] rev 12927
[doc] Remove zone cube which causes to crash on install The zone cube is not compatible with python3.
(0) -10000 -3000 -1000 -300 -100 tip