Optimize Software Fallback Path with ISA-L Integration

[asonje]

Summary

Currently, zlib-accel provides hardware acceleration through QAT and IAA with fallback to standard zlib software implementation. This creates a significant performance gap when hardware acceleration is unavailable. We propose integrating Intel ISA-L (Intelligent Storage Acceleration Library) as an intermediate performance layer to optimize the software fallback path.

Background

Zlib-accel's current fallback hierarchy:

1. Hardware acceleration (QAT/IAA) - Best performance
2. Software zlib - Standard performance with significant gap from hardware

The performance drop when falling back to software zlib can be substantial, potentially negating the benefits of using zlib-accel in mixed workload scenarios or systems without available hardware accelerators.

Proposed Solution

Integrate ISA-L's optimized compression implementation as an intermediate fallback layer:

1. Hardware acceleration (QAT/IAA) - Best performance
2. ISA-L optimized software - Enhanced performance using SIMD instructions
3. Standard zlib - Compatibility fallback

Benefits

• Reduced performance gap: ISA-L can provide significant performance improvement over standard zlib
• Broader acceleration coverage: Systems without QAT/IAA still benefit from optimized compression
• Consistency: Better performance consistency across different hardware configurations
• Maintained compatibility: ISA-L maintains zlib API compatibility

Implementation Considerations

• Fallback logic: Modify the current fallback mechanism to include ISA-L as intermediate layer
• Configuration options: Allow users to configure fallback preferences
• Performance profiling: Benchmark ISA-L performance across different workloads

Questions for Discussion

1. Should ISA-L integration be optional (compile-time flag) or always included?
2. Should there be configuration options to skip ISA-L and fall directly to standard zlib?

References

ISA-L igzip implementation

Expected Impact

This enhancement would make zlib-accel more attractive for deployment in heterogeneous environments and provide consistent performance benefits even when hardware acceleration is unavailable, ultimately improving the overall value proposition of the library.