So, for the mongopool MongoDB database driver, I'd like to create a set of performance benchmarks so that the impact of future code changes can be relatively measured.
There is, AFAIK, no real means of reliably "mocking" a MongoDb server (esp. since I'm wanting to mock the performance characteristics, not just the API.)
So, I'm thinking:
- Create a docker-compose YAML file for a container on the repo under a /performance directory. In it include:
- A specific copy of MongoDb
- A generic ubuntu instance that pulls in and runs a script that:
- runs test executable that saves a temporary text file
- runs a nim webserver that serves that text file
- Create a "performance" task in the nimble that:
- compiles the test executable; moves it to the /performance directory.
- compiles the mini webserver; moves it also.
- Builds the docker instance
- starts the docker
- pulls the results from the webserver that eventually appears; and saves the results in a file with timestamp in the name
- shuts down the docker instance and deletes it. (forcing a fresh empty virtual drive for the next MongoDB server.)
- displays the % of performance difference on the screen
Then, one simply runs nimble performance on a machine that supports docker and, many many minutes later, you have some new results.
Two huge problems with this approach:
- I'm doing a lot of re-inventing for this to work.
- This is only really meaningful when run from the same machine.
In fact, now that I've written this all out, I'm somewhat less motivated to do all this.
Does anyone have any suggestions for making this easier and more consistent?