kata-containers/tests/metrics/report

Kata Containers metrics report generator

The files within this directory can be used to generate a "metrics report" for Kata Containers. The primary workflow consists of two stages:

1. Run the provided report metrics data gathering scripts on the system(s) you wish to analyze.
2. Run the provided report generation script to analyze the data and generate a report file.

Data gathering

Data gathering is provided by the grabdata.sh script. When run, this script executes a set of tests from the tests/metrics directory. The JSON results files will be placed into the tests/metrics/results directory. Once the results are generated, create a suitably named subdirectory of tests/metrics/results, and move the JSON files into it. Repeat this process if you want to compare multiple sets of results. Note, the report generation scripts process all subdirectories of tests/metrics/results when generating the report.

Note: By default, the grabdata.sh script tries to launch some moderately large containers (i.e. 8Gbyte RAM) and may fail to produce some results on a memory constrained system.

You can restrict the subset of tests run by grabdata.sh via its commandline parameters:

Option	Description
-a	Run all tests (default)
-d	Run the density tests
-h	Print this help
-s	Run the storage tests
-t	Run the time tests

Report generation

Report generation is provided by the makereport.sh script. By default this script processes all subdirectories of the tests/metrics/results directory to generate the report. To run in the default mode, execute the following:

$ ./makereport.sh 

The report generation tool uses Rmarkdown, R and Pandoc to produce a PDF report. To avoid the need for all users to set up a working environment with all the necessary tooling, the makereport.sh script utilises a Dockerfile with the environment pre-defined in order to produce the report. Thus, you need to have Docker installed on your system in order to run the report generation. The resulting metrics_report.pdf is generated into the output subdirectory of the report directory.

Debugging and development

To aid in script development and debugging, the makereport.sh script offers a debug facility via the -d command line option. Using this option will place you into a bash shell within the running Dockerfile image used to generate the report, whilst also mapping your host side R scripts from the report_dockerfile subdirectory into the container, thus facilitating a "live" edit/reload/run development cycle. From there you can examine the Docker image environment, and execute the generation scripts. E.g., to test the scaling.R script, you can execute:

$ makereport.sh -d
# R 
> source('/inputdir/Env.R') 
> source('/scripts/lifecycle-time.R')

You can then edit the report_dockerfile/lifecycle-time.R file on the host, and re-run the source('/scripts/lifecycle-time.R') command inside the still running R container.