This benchmark is composed of 6 tests. Each test checks the 4 CRUD operations (Create, Retrieve, Update and Delete) and storage efficiency (by measuring the database size). Every operation is run in two batch size modes (large and small retrieval and transaction size). In total, when counting all the combinations of tests/operations/modes this benchmark includes 58 performance tests and 6 storage efficiency checks.
In the Result Page you can see the results of every one of these tests, as well as averages of all or a subset of these tests.
This benchmark is composed of the following tests:
Basic Person Test
The basic test uses JPA with one entity class that represents a person. The
Personclass contains an auto generated
Integerprimary key field, 10 String persistent fields, 3 date persistent fields and one int persistent field, representing the person details. This simple object model of one entity class does not include inheritance, collections, relationship or indexes.
Element Collection Test
The Element Collection Test is based on the Basic Person test with the following change. Instead of a simple
Stringfield for the phone number, a
List<String>collection field containing 1-3 phone numbers is used. This tiny change in the object model has a larger effect on the ORM/DBMS side, because of the flat nature of records in ordinary RDBMS tables.
The Inheritance test is based on the Basic Person test with the following change. Instead of one
Personentity class, an hierarchy of 3 classes is used. The original
Personfields are divided between the 3 classes in the inheritance hierarchy.
The Indexing test is based on the Basic Person test with one addition. An index is defined for one of the the String fields (the person's last name). Because JPA does not define a standard annotation for defining an index, vendor annotations are used. EclipseLink was excluded from this test because it does not provide an annotation for defining an index.
Graph (Binary Tree) Test
The Graph test uses JPA with a different object model that represents a binary tree. The
Nodeentity class, which represents a node in the tree, includes an
Integerprimary key field, two persistent reference fields to child nodes and a simple
intpersistent field with data. Each tree contains 100 nodes.
The Multithreading test is identical to the Basic Person test except that all the operations are run using 5 concurrent threads (rather than in a single thread as the other tests).
Each test executes the following 4 CRUD operations:
Persisting Entity Objects
Every test starts by populating the database. 100,000 entity objects with randomized data are persisted to the database. This is the most time consuming operation in the benchmark because persisting objects to the database using some of the tested DBMS/ORM combinations is very slow (otherwise we could have used larger databases).
Retrieving Entity Objects
After populating the database, retrieval of entity objects from the database is tested. Objects are retrieved in this step from the database by using the primary key (random primary key values are selected for retrieval).
Updating Entity Objects
The next tested CRUD operation is update. Random entity objects are retrieved from the database (by their primary key). The objects are updated in memory and then the updates are committed to the database.
Removing Entity Objects
The last tested CRUD operation is remove. Random entity objects are retrieved from the database (by their primary key) and then deleted from the database.
Retrieval by primary key (which is described above) uses JPQL query (because
is limited to a single entity object).
The Person based tests (all but the Graph/Binary Tree test), includes an additional JPQL query on a simple String field (the person's last name). The tested query retrieves all the persons whose last name starts with a specified random prefix.
Retrieval by string prefix is very common (for example, in AJAX based web applications with auto complete forms).
The disk space that a database consumes is measured when the database population is completed. Only results of this check for embedded databases are published, since disk space measurement of databases in server mode by this benchmark is inaccurate.
Batch Modes (Small / Large)
All the tests have been run in the following two batch mode:
Small Transaction Size / Retrieval Size
In this mode transactions are committed after every 5 insert/update/delete operations and entity objects are retrieved in groups of 5 entity objects. Except in the Graph/Binary Tree test, in which every single tree has 100 nodes, so 100 is used instead of 5.
Large Transaction Size / Retrieval Size
In this mode transactions are committed after every 5,000 insert/update/delete operations and entity objects are retrieved in groups of 5,000 entity objects.