Skip to content

[Version 3] New heuristic for RAM #804

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
merged 19 commits into from
Apr 13, 2025
Merged

Conversation

benoit-cty
Copy link
Contributor

Breaking change in RAM power estimation.

In V3, we need to improve the accuracy of the RAM power estimation.
Because the power consumption of RAM is not linear with the amount of memory used,
for example, in servers you could have thousands of GB of RAM but the power
consumption would not be proportional to the amount of memory used, but to the number
of memory modules used.
But there is no way to know the memory modules used in the system, without admin rights.
So we need to build a heuristic that is more accurate than the previous one.
For example keep a minimum of 2 modules. Execept for ARM CPU like rapsberry pi where we will consider a 3W constant.
Then consider the max RAM per module is 128GB and that RAM module only exist in power of 2 (2, 4, 8, 16, 32, 64, 128).
So we can estimate the power consumption of the RAM by the number of modules used.

1. **ARM CPU Detection**:
- Added a `_detect_arm_cpu` method that checks if the system is using an ARM architecture
- For ARM CPUs (like Raspberry Pi), a constant 3W will be used as the minimum power

2. **DIMM Count Estimation**:
- Created a `_estimate_dimm_count` method that intelligently estimates how many memory modules might be present based on total RAM size
- Takes into account that servers typically have more and larger DIMMs
- Assumes DIMM sizes follow powers of 2 (4GB, 8GB, 16GB, 32GB, 64GB, 128GB) as specified

3. **Scaling Power Model**:
- Base power per DIMM is 2.5W for x86 systems and 1.5W for ARM systems
- For standard systems (up to 4 DIMMs): linear scaling at full power per DIMM
- For medium systems (5-8 DIMMs): decreasing efficiency (90% power per additional DIMM)
- For large systems (9-16 DIMMs): further reduced efficiency (80% power per additional DIMM)
- For very large systems (17+ DIMMs): highest efficiency (70% power per additional DIMM)

4. **Minimum Power Guarantees**:
- Ensures at least 5W for x86 systems (assuming 2 DIMMs at minimum)
- Ensures at least 3W for ARM systems as requested

### Example Power Estimates:

- **Small laptop (8GB RAM)**: ~5W (2 DIMMs at 2.5W each)
- **Desktop (32GB RAM)**: ~10W (4 DIMMs at 2.5W each)
- **Small server (128GB RAM)**: ~18.6W (8 DIMMs with efficiency scaling)
- **Large server (1TB RAM)**: ~44W (using 16x64GB DIMMs with high efficiency scaling)

This approach significantly improves the accuracy for large servers by recognizing that RAM power consumption doesn't scale linearly with capacity, but rather with the number of physical modules. Since we don't have direct access to the actual DIMM configuration, this heuristic provides a more reasonable estimate than the previous linear model.

The model also includes detailed debug logging that will show the estimated power for given memory sizes, helping with validation and fine-tuning in the future.

@benoit-cty
Copy link
Contributor Author

For the record, here is the proposition of DeepSeek R1 :
To improve the V3 heuristic for RAM power estimation while accounting for older systems with smaller modules, the following approach can be used:
Heuristic Adjustments:

ARM Exception: For ARM-based systems (e.g., Raspberry Pi), use a fixed 3W.

Minimum Modules: Assume a minimum of 2 modules for non-ARM systems to avoid underestimation in small configurations.

Dynamic Max Module Size:

    Use a lower assumed maximum module size (e.g., 32GB) instead of 128GB. This increases the estimated number of modules for older systems, improving accuracy without admin data.

    Calculate modules as:
    Number of Modules = max(ceil(Total RAM / 32GB), 2)
    This balances newer systems (which may use larger modules) and older ones (which use smaller ones).

Per-Module Power: Assign 3W per module, reflecting typical power usage regardless of module size.

Example Calculations:

16GB System:
16GB / 32GB = 0.5 → ceil(0.5) = 1 → 2 modules (min enforced) → 6W.

64GB System:
64GB / 32GB = 2 → 2 modules → 6W (aligns with newer systems using 32GB modules; older systems with 8GB modules would have 8 modules → 24W, but heuristic estimates conservatively).

128GB System:
128GB / 32GB = 4 → 4 modules → 12W.

Trade-offs:

Older Systems: Closer estimates for configurations using smaller modules (e.g., 8GB/module) by assuming 32GB/module.

Newer Systems: Slight overestimation for large modules (e.g., 64GB/module), but more accurate than the linear model.

This approach prioritizes accuracy for older systems while maintaining simplicity and avoiding reliance on admin-only data.

@benoit-cty benoit-cty marked this pull request as ready for review April 6, 2025 09:22
@benoit-cty benoit-cty requested a review from Copilot April 11, 2025 12:59
Copy link

@Copilot Copilot AI left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Copilot reviewed 41 out of 48 changed files in this pull request and generated 1 comment.

Files not reviewed (7)
  • docs/.buildinfo: Language not supported
  • docs/_sources/examples.rst.txt: Language not supported
  • docs/_sources/methodology.rst.txt: Language not supported
  • docs/_sources/parameters.rst.txt: Language not supported
  • docs/edit/examples.rst: Language not supported
  • docs/edit/methodology.rst: Language not supported
  • docs/edit/parameters.rst: Language not supported
Comments suppressed due to low confidence (2)

docs/examples.html:172

  • The word 'begginning' appears to be misspelled; consider changing it to 'beginning'.
You instantiate the tracker and call the <cite>start()</cite> method at the begginning of the Notebook.

codecarbon/output_methods/file.py:55

  • Consider if the removal of the dropna call is intentional; without it, columns with only NaN values may persist in the CSV output.
new_df = new_df.dropna(axis=1, how="all")

@benoit-cty benoit-cty force-pushed the feat/ram_stick_power branch from 4ac0901 to 90d6ec9 Compare April 13, 2025 18:30
@@ -66,14 +66,17 @@ def is_rapl_available() -> bool:
def is_psutil_available():
try:
nice = psutil.cpu_times().nice
if nice > 0.1:
if nice > 0.0001:
Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Because on huge CPU, it could be above 0 but under 0.1.

Copy link
Collaborator

@SaboniAmine SaboniAmine left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Merci Benoît !

@@ -61,7 +63,10 @@ class Energy:

@classmethod
def from_power_and_time(cls, *, power: "Power", time: "Time") -> "Energy":
return cls(kWh=power.kW * time.hours)
assert isinstance(power.kW, float)
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Quel est l'intérêt de l'assert ici ? ça ne risque pas de casser le programme à un endroit non désiré ?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Cette endroit est la base du calcul, si jamais il n'y a pas la bonne unité il y a plein d'effets de bord.
Ca m'est arrivé lors du dev et j'ai mis du temps à trouver la source du problème. Je me suis dit qu'en le vérifiant ici, cela évitera des problèmes à l'avenir.

# Minimum 5W for x86 as requested (2 sticks at 2.5W)
min_power = base_power_per_dimm * 2

# Estimate power based on DIMM count with decreasing marginal power per DIMM as count increases
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

ça pourrait être intéressant de faire des statistiques sur les setups les plus courants. Peut être via les données remontées par l'API

@benoit-cty benoit-cty merged commit 8eac358 into codecarbon_v3_rc Apr 13, 2025
4 checks passed
@benoit-cty benoit-cty deleted the feat/ram_stick_power branch April 13, 2025 20:10
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants