changeset | 6cd938c29ca3 |
---|---|
branch | default |
bookmark | |
tag | tip |
user | Philippe Pepiot <ph@itsalwaysdns.eu> |
description | [server] Make connection pooler configurable and set better default values Drop the configuration connections-pool-size and add new configurations options: * connections-pool-min-size. Set to 0 by default so we open connections only when needed. This avoid opening min-size*processes connections at startup, which is, it think, a good default. * connections-pool-max-size. Set to 0 (unlimited) by default, so we move the bottleneck to postgresql. * connections-idle-timeout. Set to 10 minutes. I don't have arguments about this except that this is the default in pgbouncer. |
files | cubicweb/server/repository.py cubicweb/server/serverconfig.py cubicweb/test/unittest_cwconfig.py doc/changes/3.28.rst |
changeset | a6d19bc7fed3 |
---|---|
branch | |
bookmark | |
tag | |
user | Philippe Pepiot <ph@itsalwaysdns.eu> |
description | [server] Enhance connections-pooler-enabled documentation |
files | cubicweb/server/serverconfig.py |
changeset | 85f0fe1bb78b |
---|---|
branch | |
bookmark | |
tag | |
user | Philippe Pepiot <ph@itsalwaysdns.eu> |
description | [server] move connection pooler initialization logic to get_cnxset() This avoid complex logic in Repository initialization. |
files | cubicweb/server/repository.py |
changeset | dd9e98b25213 |
---|---|
branch | |
bookmark | |
tag | |
user | Philippe Pepiot <ph@itsalwaysdns.eu> |
description | [server] dynamically close idle database connections When pool hasn't been empty for `idle_timeout` time, start closing connections. |
files | cubicweb/server/repository.py |
changeset | fa0cd558d829 |
---|---|
branch | |
bookmark | |
tag | |
user | Philippe Pepiot <ph@itsalwaysdns.eu> |
description | [server] implement dynamic database pooler Opening too much database connection has a cost at startup and also PostgreSQL as a maximum number of connection (100 by default). This get worse when starting multiple wsgi processes, since each process has its own database pool. Instead of opening `connections-pool-size` connections to the database at startup, just open one and open more only when needed. |
files | cubicweb/server/repository.py |
changeset | 01810941a4be |
---|---|
branch | |
bookmark | |
tag | |
user | Philippe Pepiot <ph@itsalwaysdns.eu> |
description | [server] use a LifoQueue in _CnxSetPool In postgresql, some cache is attached to the connection. Using a LifoQueue (last-in, first-out) makes a few connections to get the most load which give best performance. |
files | cubicweb/server/repository.py |
changeset | 2c3a8a4f5a02 |
---|---|
branch | |
bookmark | |
tag | |
user | Philippe Pepiot <ph@itsalwaysdns.eu> |
description | [server] extract creating a new cnxset in a _new_cnxset() helper So we can move logic specific to _CnxSetPool here. |
files | cubicweb/server/repository.py |
changeset | 39ee325e6758 |
---|---|
branch | |
bookmark | |
tag | |
user | Philippe Pepiot <ph@itsalwaysdns.eu> |
description | [server] avoid a possible race condition on _CnxSetPool.close() The pool could become empty between time to check and time to use. |
files | cubicweb/server/repository.py |
changeset | 3667f6df1ec3 |
---|---|
branch | |
bookmark | |
tag | |
user | Philippe Pepiot <ph@itsalwaysdns.eu> |
description | [server] extract "no pooler" CnxSet class to a _BaseCnxSet class So we get rid of "if self._queue is None" in each method of _CnxSetPool Also add helper get_cnxset(source, size) to instantiate the correct pooler class. |
files | cubicweb/server/repository.py |
changeset | 0c973204033a |
---|---|
branch | |
bookmark | |
tag | |
user | Philippe Pepiot <ph@itsalwaysdns.eu> |
description | [server] prevent returning closed cursor to the database pool In since c8c6ad8 init_repository use repo.internal_cnx() instead of repo.system_source.get_connection() so it use the pool and we should not close cursors from the pool before returning it back. Otherwise we may have "connection already closed" error. This bug only trigger when connection-pool-size = 1. Since we are moving to use a dynamic pooler we need to get this fixed. This does not occur with sqlite since the connection wrapper instantiate new cursor everytime, but this occur with other databases. |
files | cubicweb/server/__init__.py |