Estimate the environmental impacts of live apps on premise
10 January, 2023
This article is the second part of a serie. See Intro here.
⚠ DRAFT ⚠
Possible plan of article :
- Context / difficulties
- Methodological approaches
- Tools
- Limits and emerging approaches
- Global limits
- Not one single tool yet
- How to aggregate data / Automate collection
Context
This article presents tools we use to estimate environmental impacts of Backend applications.
Assumptions:
- we operate the hardware (it can be virtualized, but we have access to the virtualization layer)
- hardware can be on premise or in rented data center or rented as bare metal in a non accessible DC.
- we want to estimate the environmental impacts in live conditions (i.e. users are using the service).
All the tools presented, require that we have low level access to the hardware and to the first layer of software (OS or Virtualization layer).
This excludes the scenario where we are using virtualized infrastructure as a service (IAAS). However theses approaches will work in the case where we use bare metal instances, even if provided as a service.
Measure based approaches
We can adopt a measure-based approach. In practice we are often limited to measure the Energy consumed by our IT. We can measure it globally (top-down) or locally (bottom-up).
In both case we need to find a way to attribute the energy consumption to a specific device or application. This implies a precise inventory of our physical assets and services.
Power meters
A first possibility is to use physical power meters. They can be attached to a DataCenter, a rack or even a physical server.
Pros:
- we can measure all part of the system (or even use our electricity bill)
- the results are precise
Cons:
- expensive to install and operate.
- the results we get are global . It leaves us with the problem of attribution to a specific service
- it accounts only for the power (Electricity mainly) related impacts.
Software meters
Recent hardware come with meters that monitor power usage and can be queried by software.
Research paper show for example that the global electrical consumption of servers is closely linked to the consumption of CPU and RAM. So theses figures are sufficient to estimate the server consumption.
Pros:
- easy to install
- fine granularity of measure (ability to measure per server, and to estimate to a container or process level)
Cons:
- they do not account for the whole chain. The energy used to operate the DC globally or specific functions like network or storage is generally excluded.
- they work only on supported systems and supported hardware
- they do not account for all type of specialized hardware (like GPUs).
- the precision of the measure is questioned
Main tools:
- RAPL software Meters :
Hybrid with model-based approaches
Mixed approach where we measure what we can (genrally workload) from the OS and the remaining impacts from a model. This brings the added value of having a multicriteria approach (like in Boavizta dataset).
- Boavizta API work GitHub - Boavizta/boaviztapi: 🛠 Giving access to BOAVIZTA reference data and methodologies trough a RESTful API
- Boagent: GitHub - Boavizta/boagent: Local API and monitoring agent focussed on environmental impacts of the host.
- Work from UPPA (measure -> model): Joular Project
- EBPF based:
- GitHub - sustainable-computing-io/kepler: Kepler (Kubernetes-based Efficient Power Level Exporter) uses eBPF to probe energy related system stats and exports as Prometheus metrics
- kepler/OSS-NA22.pdf at main · sustainable-computing-io/kepler
- GitHub - rolandobrondolin/DEEP-mon: an eBPF-based monitoring tool to measure container resource usage, power consumption, network I/O, and file I/O