Skip to content

multifacet/platform-workloads

Folders and files

NameName
Last commit message
Last commit date

Latest commit

 

History

22 Commits
 
 
 
 
 
 
 
 
 
 

Repository files navigation

Platform Workloads 🚀

A collection of workloads to run across different isolation platforms (linux containers, gVisor and firecracker microVMs).

Workload Categories


Microbenchmarks

Quick Start

Build and run individual microbenchmarks for linux containers and gVisor:

cd microbenchmarks/<benchmark_name>
docker build -t <benchmark_name>_micro .

Available Microbenchmarks

CPU-intensive workloads using Sysbench prime number computation.

Comprehensive stress testing tool for CPU, memory, I/O, and system components.

Memory allocation and access pattern benchmarks with configurable allocation patterns.

File system metadata operations using FileBench (create, delete, list operations).

How to Run

Each microbenchmark can be run independently on runc or gVisor(runsc):

# Example: Run CPU microbenchmark
cd microbenchmarks/cpu
docker run -itd --name="worker" --runtime=runc --cpuset-cpus=0 --memory=512m sysbench_micro
docker exec -itd worker /app/run_sysbench.sh 1 20000 1 300

# Example: Run stress-ng microbenchmark
cd microbenchmarks/stress_ng
docker run -itd --name="stressng_worker" --runtime=runc --cpuset-cpus=0 --memory=512m stressng_micro
docker exec -itd stressng_worker /usr/local/bin/run_stressng.sh --stressor cpu --workers 1 cpu.csv 300 --log

# Example: Run memory microbenchmark
cd microbenchmarks/memory
docker run -itd --name="worker" --runtime=runc --cpuset-cpus=0 --memory=512m mem_micro
docker exec -itd worker /usr/local/bin/memory.sh --log 21 13 3 2 300

See individual microbenchmark README files for detailed parameters and usage patterns. For Firecracker microVMs, you will need to copy appropriate binaries and scripts to the microVM and run them directly inside the microVM environment.


FaaS Workloads

Note: These workloads are adopted from FunctionBench and modified to run locally without external dependencies.

Quick Start

Extract datasets and build containers:

cd faas
./setup_datasets.sh
./build_docker_containers.sh

Available Workloads

Image manipulation and computer vision functions.

Machine learning model training on text data.

Pre-trained model inference serving (CNN, RNN, Face Detection).

Video manipulation and encoding functions.

See individual workload directories for run instructions and parameters.


Cloud Workloads

Note: These workloads are adopted from CloudSuite and converted to standalone mode for our experiments.

We provide pre-built Docker images for all cloud workloads (for use with runc and runsc-kvm). For Firecracker (fc), see the Firecracker Image Setup section below.

Quick Start

Download pre-built Docker images (runc / runsc-kvm):

./cloud-workloads/docker_image_setup.sh

Available Workloads

Hadoop-based data processing with Mahout machine learning on Wikipedia datasets.

Memcached-based in-memory caching performance testing.

Cassandra NoSQL database serving with YCSB benchmarking.

Spark-based graph processing on Twitter social network data.

Spark MLlib collaborative filtering on MovieLens datasets.

NGINX-based video streaming with performance testing.

How to Run

Each workload can be run independently:

# Example: Run data analytics workload
cd cloud-workloads/dataAnalytics
docker run -id --rm --name worker --cpuset-cpus="0-3" --memory="24g" mydataanalytics
docker exec worker /root/entrypoint.sh

See individual workload README files for specific parameters and instructions.


Datasets

Large datasets for cloud and FaaS workloads are available as GitHub Release assets:

  • GitHub Releases — pre-packaged datasets for each workload (wiki data, Twitter graph, MovieLens, video files, FaaS datasets, etc.)

Alternatively, datasets can be obtained directly from the sources:


Firecracker Image Setup

Firecracker (fc) experiments require a pre-built ext4 root filesystem for each workload. The firecracker-setup repository contains scripts for booting Firecracker VMs, resizing images, and using chroot to populate images without booting a VM. Follow the common steps below, then refer to each workload's README for the workload-specific installation steps.

1. Get a Base Image

Download a base Ubuntu root filesystem and kernel following the Firecracker getting-started guide.

2. Resize the Image

Each workload specifies its required image size. See firecracker-setup for resize scripts.

3. Boot and SSH In

Boot the VM using the scripts in firecracker-setup, or use chroot to populate the image directly without booting.

4. Install Workload Software

  • Cloud workloads: Follow the Building the Firecracker Image section in each workload's README. The cloud-workloads/commons/ directory contains Dockerfiles and install scripts that mirror what is installed inside each image (Java, Hadoop, Cassandra, Spark, Memcached).
  • Microbenchmarks: The Dockerfiles and scripts under microbenchmarks/ can be used as a reference to install the required binaries (sysbench, stress-ng, filebench, etc.) inside the microVM.
  • FaaS workloads: The Dockerfiles and requirements.txt files under faas/ describe the Python dependencies and model files needed to run each function inside the microVM.

Repository Structure

platform-workloads/
├── cloud-workloads/                # Cloud application workloads
│   ├── docker_image_setup.sh       # Download and tag pre-built Docker images
│   ├── stress_cloud_subsystems.sh  # Performance stress testing driver
│   ├── trace_cloud_subsystems.sh   # Lock tracing driver (LIMA)
│   ├── workloads.yaml              # Central configuration for all workloads
│   ├── commons/                    # Shared Dockerfiles and install scripts
│   ├── dataAnalytics/              # Hadoop + Mahout workload
│   ├── dataCaching/                # Memcached workload
│   ├── dataServing/                # Cassandra + YCSB workload
│   ├── graphAnalytics/             # Spark graph processing workload
│   ├── inMemoryAnalytics/          # Spark MLlib workload
│   └── mediaStreaming/             # NGINX video streaming workload
├── faas/                           # Function-as-a-Service workloads
│   ├── build_docker_containers.sh  # Build all FaaS containers
│   ├── setup_datasets.sh           # Extract compressed datasets
│   ├── image_processing/           # Image manipulation functions
│   ├── model_training/             # ML model training functions
│   ├── model_serving/              # ML model inference functions
│   └── video_processing/           # Video processing functions
└── microbenchmarks/                # System performance microbenchmarks
    ├── cpu/                        # CPU Sysbench benchmarks
    ├── stress_ng/                  # Stress-ng comprehensive testing
    ├── memory/                     # Memory allocation benchmarks
    └── file_metadata/              # File system metadata benchmarks

About

No description, website, or topics provided.

Resources

Stars

Watchers

Forks

Packages

 
 
 

Contributors