Implement HikariCP as a replacement for c3p0

Description

Copying from https://github.com/idempiere/idempiere/pull/926

Hi all!

I order to support PostgreSQL reconnecting after failover when using client side node selection I replaced C3P0 with HikariCP. HikariCP is an actively maintained, Apache licensed connection pool with substantially better performance and better resilience to failure (DB disconnects, etc.) then C3P0. Read more about it here: https://github.com/brettwooldridge/HikariCP (including comparison and benchmarks with C3P0). During the development of HikariCP a lot of care was spend on making HikariCP behave as correctly and predictable as possible.

Let me know what you think!

Aside from HikariCP the following changes were made:

Cleaned up the getCachedConnection method. With HikariCP there is no need to retry to obtain a connection since getting an connection will block until a free connection is available or until a timeout is reached (default 30 seconds) at which point an SQLException is thrown. This also removed calling Runtime.getRuntime().runFinalization();. HikariCP is currently configured to detect / log leaks when a connection hasn't returned to the pool for longer then 5 minutes.

Initialization of the datasource must happen in the getDataSource() method because at object construction not all JDBC config is known. However this method could (as far as I could tell) be called concurrently from multiple threads but had no mechanism to prevent initializing the DB pool multiple times. The variable in which the pool itself was stored (m_ds) also was not marked volatile or immutable which could lead to visibility issues. Instead of lazy initialization of the pool in the getDataSource() method the pool could probably better be initialized at object construction. However I wasn't able to achieve that without breakage therefor I made the initialization mechanism work correctly with concurrent invocations.

The MaxStatementsPerConnection option was removed because HikariCP doesn't support it.

Mattermost discussion:
https://mattermost.idempiere.org/idempiere/pl/fxubzbobxtnt7czaxor8z7bf6r

As discussed on Mattermost this PR now only includes the change from C3P0 to HikariCP. I have removed the IDEMPIERE_DB_JDBC_URL.

Environment

None

Attachments

1

Activity

Show:

Jasper SiepkesSeptember 7, 2022 at 11:51 AM

Does my suggestion address your concern or do you feel we need to go another way about it?

Carlos RuizSeptember 1, 2022 at 1:58 PM

ah - you mean with the exact same jar name - I didn’t try that - it must work

Jasper SiepkesSeptember 1, 2022 at 1:51 PM

I have the feeling that removing the jar file manually on the operating system can have other consequences

I suspect removing the jar from the base and then re-inserting the C3P0 jar in to base would be the way to go.

As far is Equinox is concerned nothing has changed because the plugin ID, name, etc has not changed. Therefor any reference Equinox might have to the plugin will still work (like the file defining runlevels for example).

Carlos RuizSeptember 1, 2022 at 1:22 PM

Thanks

Must be something related to the “base” plugins, those that are installed at the installer.

From what I see:

  • Felix and OSGi console install the plugins in a file, somewhere like: configuration/org.eclipse.osgi/[plugin_number]/0/bundleFile

  • The plugins installed in the base are located in the plugins folder

  • From OSGi console you can install/uninstall the plugins installed in the configuration/org.eclipse.osgi folder

  • But, from what I see here, it seems I cannot uninstall from OSGi console the plugins from the plugins folder

  • So, wondering what would be the right way to uninstall a base plugin. I have the feeling that removing the jar file manually on the operating system can have other consequences

Jasper SiepkesSeptember 1, 2022 at 1:02 PM

for some reason it cannot be removed using the OSGi console as it appears again on restart

My memory is a bit hazy on the subject but as far as I know you can’t “delete“ plugins from the OSGi console (meaning deleting the actual JAR) with neither Felix nor Equinox. You can only transition their lifecycle to “uninstalled“. But as soon as you restart the OSGi runtime it will load it’s bundles as it normally would. There is no “persistence“ of the commands you enter in the OSGi console in that regard.

Carlos RuizSeptember 1, 2022 at 10:12 AM

Well - I did a test and it worked partially fine:

  • generated plugin org.compiere.db.postgresql.provider from current master (c3p0)

  • deployed a server with the hikaricp version

  • started the server

  • using OSGi console I tried to uninstall the org.compiere.db.postgresql.provider (hikaricp) and install the plugin from step 1

    • this didn’t work, the old plugin survives a restart

  • manually I went to idempiere home and removed the file plugins/org.compiere.db.postgresql.provider*.jar

  • restarted the server - and now the server is running c3p0 instead of hikaricp

So, it seems possible, currently the database is “injected” using the interface org.compiere.db.AdempiereDatabase in OSGI-INF/pgprovider.xml and the interface org.adempiere.install.IDatabaseConfig on plugin.xml

The only thing that I didn’t like is the need to remove the plugin manually, for some reason it cannot be removed using the OSGi console as it appears again on restart, and the new installed plugin is not used.

Heng Sin LowSeptember 1, 2022 at 12:33 AM

To do that, would need to add a new level of abstraction. Perhaps, a new OSGi datasource interface and you can have implementation for c3p0, hikaricp, etc.

Carlos RuizAugust 31, 2022 at 4:04 PM

the idea about external plugins is that they have their own repo

Jasper SiepkesAugust 31, 2022 at 3:42 PM

I’m not really qualified to say what such a solution would look like. For example would the plugin be in a separate repo or in the main repo? I think is probably best qualified to say something about the technical side of it?

Carlos RuizAugust 30, 2022 at 5:19 PM

The thing is that we are replacing an 11 years plugin tested in really heavy conditions.

So, for me it creates some uncertainty to replace it with something that needs to prove to be so reliable as it has been c3p0, so, I would like to have a backdoor in case some scenarios are not well covered by HikariCP.

Some iDempiere implementations (as framework) run millions of transactions per day, and so far c3p0 has been very capable of managing such loads.

Jasper SiepkesAugust 30, 2022 at 2:05 PM

Hi !

Might be possible but what use-case could someone have to prefer c3p0 and invest the time to maintain such a plugin? There is already a bunch of code duplication between the Oracle and PostgreSQL plugin. With a c3p0 plugin that duplication doubles.

Fixed

Details

Assignee

Reporter

Fix versions

Priority

Created October 23, 2021 at 1:40 PM
Updated March 1, 2024 at 4:25 PM
Resolved January 3, 2024 at 11:16 AM