offer: make the merkle tree signature public #3892

vincenzopalazzo · 2025-06-26T10:30:20Z

This is helpful for users who want to use the Merkle tree signature in their own code, for example, to verify the signature of bolt12 invoices or recreate it.

Very useful for people who are building command line tools for the bolt12 offers.

I am opening this, but I do not know if it is something that you want to do, but at the same time, IMHO, this is very useful to expose because it allows to use LDK in command line tools and in learning tools.

However, this is not strictly necessary because Bolt12Invoice::try_from already verifies the signature, but I would open this to know your point of view.

ldk-reviews-bot · 2025-06-26T10:30:22Z

👋 Thanks for assigning @valentinewallace as a reviewer!
I'll wait for their review and will help manage the review process.
Once they submit their review, I'll check if a second reviewer would be helpful.

This is helpfull for the users that want to use the merkle tree signature in their own code, for example to verify the signature of bolt12 invoices or recreate it. Very useful for people that are building command line tools for the bolt12 offers. Signed-off-by: Vincenzo Palazzo <[email protected]>

dunxen · 2025-06-26T11:51:14Z

I'm not personally opposed to this and it may be nice to expose it for the use cases you mention.
However, maybe the doc comments for these should be updated to indicate that these don't need to be used directly and they're really for internal use, but made public for use in applications like you specified. We should redirect them to the appropriate methods for production use.

I'm also not sure about any API guarantees for these methods. I don't think it's too much of a hassle to treat them the same way we treat any other public API.

codecov · 2025-06-26T15:08:45Z

Codecov Report

Attention: Patch coverage is 93.82716% with 10 lines in your changes missing coverage. Please review.

Project coverage is 88.80%. Comparing base (0fe51c5) to head (29e5bd6).
Report is 57 commits behind head on main.

Files with missing lines	Patch %	Lines
lightning/src/offers/merkle.rs	90.00%	9 Missing and 1 partial ⚠️

Additional details and impacted files

@@            Coverage Diff             @@
##             main    #3892      +/-   ##
==========================================
- Coverage   89.66%   88.80%   -0.87%     
==========================================
  Files         164      166       +2     
  Lines      134661   119416   -15245     
  Branches   134661   119416   -15245     
==========================================
- Hits       120743   106045   -14698     
+ Misses      11237    11034     -203     
+ Partials     2681     2337     -344

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:

❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.

valentinewallace · 2025-06-27T16:20:23Z

lightning/src/offers/merkle.rs

@@ -41,7 +41,7 @@ impl TaggedHash {
 	/// Creates a tagged hash with the given parameters.
 	///
 	/// Panics if `bytes` is not a well-formed TLV stream containing at least one TLV record.
-	pub(super) fn from_valid_tlv_stream_bytes(tag: &'static str, bytes: &[u8]) -> Self {
+	pub fn from_valid_tlv_stream_bytes(tag: &'static str, bytes: &[u8]) -> Self {


Do we have a user for this? Feels pretty low-level to be exposing. Echo dunxen's sentiments about improving the docs if we do want to expose it.

Yea, if we're gonna expose this we should really add a validating wrapper version that checks the stream is valid and can Err instead of panicking.

Do we have a user for this? Feels pretty low-level to be exposing.

Yeah, there is. Everyone who wants to verify that an offer is linked to a bolt12 invoice, also if the big picture is that the bolt12 invoice will never be exposed, but for now that we do not have a better PoP a person should allow this kind of verification IMHO

A possible example of verification can be

/// Verify that an offer, invoice, and preimage are valid and consistent pub fn verify_offer_payment(offer: &str, invoice: &str, preimage: &str) -> Result<()> { log::info!("Starting verification process..."); log::info!("Offer: {}", offer); log::info!("Invoice: {}", invoice); log::info!("Preimage: {}", preimage); let offer = Offer::from_str(offer).map_err(|e| anyhow::anyhow!("Failed to parse offer: {:?}", e))?; // Parse the bolt12 invoice preimage let (hrp, data) = bech32::decode_without_checksum(invoice)?; if hrp.as_str() != "lni" { return Err(anyhow::anyhow!( "Invalid HRP: expected 'lni', found '{}'", hrp )); } let data = Vec::<u8>::from_base32(&data)?; let invoice = Bolt12Invoice::try_from(data) .map_err(|e| anyhow::anyhow!("Failed to parse BOLT12: {e:?}"))?; let issuer_sign_pubkey = if let Some(issue_sign_pubkey) = offer.issuer_signing_pubkey() { issue_sign_pubkey } else { let valid_signing_keys = offer .paths() .iter() .filter_map(|path| path.blinded_hops().last()) .map(|hop| hop.blinded_node_id) .collect::<Vec<_>>(); log::debug!("{:?}", valid_signing_keys); let valid_signing_keys = valid_signing_keys .first() .ok_or(anyhow::anyhow!( "Offer does not contain issuer signing pubkey" ))? .clone(); valid_signing_keys }; let inv_signing_pubkey = invoice.signing_pubkey(); if issuer_sign_pubkey != inv_signing_pubkey { return Err(anyhow::anyhow!( "Offer and invoice issuer signing pubkeys do not match" )); } let mut bytes = vec![]; invoice .write(&mut bytes) .map_err(|e| anyhow::anyhow!("Failed to serialize invoice: {:?}", e))?; let tagged_hash = TaggedHash::from_valid_tlv_stream_bytes(SIGNATURE_TAG, &bytes); if let Err(err) = merkle::verify_signature(&invoice.signature(), &tagged_hash, issuer_sign_pubkey) { anyhow::bail!("Failed to verify invoice signature: {:?}", err); } let preimage = hex::decode(preimage).map_err(|e| anyhow::anyhow!("Failed to decode preimage: {:?}", e))?; let preimage: [u8; 32] = preimage .try_into() .map_err(|_| anyhow::anyhow!("Preimage must be 32 bytes"))?; let preimage = PaymentPreimage(preimage); // Validate the Proof of Payment for the invoice let computed_hash = PaymentHash::from(preimage); let payment_hash = invoice.payment_hash(); if computed_hash != payment_hash { anyhow::bail!("Fail to perform PoP"); } log::info!("Parsed offer: {:?}", offer); log::info!("Parsed invoice: {:?}", invoice); Ok(()) }

Hmm... we verify the invoice's signature when parsing it. So it's just a matter of matching that the Offer given is the same one that is part of the Bolt12Invoice. This should be possible by comparing Offer::id -- which is just the tagged hash of the offer's merkle root -- against one obtained from the Bolt12Invoice. We could possibly add Bolt12Invoice::offer_id for this.

Note that while OfferId uses an LDK-specific tag, it is computed over the TLV stream bytes so it doesn't matter if the offer wasn't created with LDK.

Ok implemented it @jkczyz cfd6ad4 let me know what do you think.

However, I still think that ldk should expose a way to verify the signature with the public key that is inside the offer, at least till we do not get a better PoP with bolt12

Ok implemented it @jkczyz cfd6ad4 let me know what do you think.

Thanks, left some comments.

However, I still think that ldk should expose a way to verify the signature with the public key that is inside the offer, at least till we do not get a better PoP with bolt12

Given the signature is verified when parsing, shouldn't it be sufficient to check that Bolt12Invoice::signing_pubkey matches the one in the offer? The parser is already doing this when calling check_invoice_signing_pubkey, too, FWIW. Not sure I understand why we need to expose these for the user to piece together and call on their own.

At very least, there doesn't seem to be a need to expose TaggedHash::from_valid_tlv_stream_bytes if we can just have a method on Bolt12Invoice give the TaggedHash for manual signature verification.

ldk-reviews-bot · 2025-06-27T16:20:29Z

👋 The first review has been submitted!

Do you think this PR is ready for a second reviewer? If so, click here to assign a second reviewer.

This commit addresses review feedback by adding a validating version of the merkle tree signature functions that returns proper error types instead of panicking on invalid input. The new from_tlv_stream_bytes function validates TLV streams before processing and returns a Result, making it safer for command line tools and external applications to use. Additionally, documentation has been improved to clarify that these are low-level functions for specific use cases, with clear guidance to use higher-level methods for production. Comprehensive tests have been added to ensure consistency between the validating and non-validating functions and prevent regressions. Signed-off-by: Vincenzo Palazzo <[email protected]>

graphite-app · 2025-07-05T10:48:13Z

lightning/src/offers/merkle.rs

+	pub fn from_tlv_stream_bytes(tag: &'static str, bytes: &[u8]) -> Result<Self, TlvStreamError> {
+		// Validate the TLV stream first
+		if bytes.is_empty() {
+			return Err(TlvStreamError::EmptyStream);
+		}
+
+		// Try to parse the TLV stream to check validity
+		let mut cursor = io::Cursor::new(bytes);
+		let mut has_records = false;
+
+		while cursor.position() < bytes.len() as u64 {
+			// Try to read type
+			let type_result = <BigSize as Readable>::read(&mut cursor);
+			if type_result.is_err() {
+				return Err(TlvStreamError::InvalidRecord);
+			}
+
+			// Try to read length
+			let length_result = <BigSize as Readable>::read(&mut cursor);
+			if length_result.is_err() {
+				return Err(TlvStreamError::InvalidRecord);
+			}
+
+			let length = length_result.unwrap().0;
+			let end_position = cursor.position() + length;
+
+			// Check if the record extends beyond the buffer
+			if end_position > bytes.len() as u64 {
+				return Err(TlvStreamError::InvalidRecord);
+			}
+
+			// Skip the value
+			cursor.set_position(end_position);
+			has_records = true;
+		}
+
+		if !has_records {
+			return Err(TlvStreamError::EmptyStream);
+		}
+
+		// If validation passes, create the tagged hash
+		Ok(Self::from_valid_tlv_stream_bytes(tag, bytes))


The TLV stream validation implementation is missing a critical requirement from the Lightning Network specification: TLV records must appear in ascending order by type.

To ensure full compliance, consider tracking the previous type value during iteration and verifying that each new type is greater than the previous one:

let mut previous_type: Option<u64> = None; while cursor.position() < bytes.len() as u64 { // Try to read type let type_result = <BigSize as Readable>::read(&mut cursor); if type_result.is_err() { return Err(TlvStreamError::InvalidRecord); } let current_type = type_result.unwrap().0; // Check ascending order if let Some(prev) = previous_type { if current_type <= prev { return Err(TlvStreamError::InvalidOrder); } } previous_type = Some(current_type); // Rest of the validation... }

This would require adding an InvalidOrder variant to the TlvStreamError enum.

Suggested change

pub fn from_tlv_stream_bytes(tag: &'static str, bytes: &[u8]) -> Result<Self, TlvStreamError> {

// Validate the TLV stream first

if bytes.is_empty() {

return Err(TlvStreamError::EmptyStream);

}

// Try to parse the TLV stream to check validity

let mut cursor = io::Cursor::new(bytes);

let mut has_records = false;

while cursor.position() < bytes.len() as u64 {

// Try to read type

let type_result = <BigSize as Readable>::read(&mut cursor);

if type_result.is_err() {

return Err(TlvStreamError::InvalidRecord);

}

// Try to read length

let length_result = <BigSize as Readable>::read(&mut cursor);

if length_result.is_err() {

return Err(TlvStreamError::InvalidRecord);

}

let length = length_result.unwrap().0;

let end_position = cursor.position() + length;

// Check if the record extends beyond the buffer

if end_position > bytes.len() as u64 {

return Err(TlvStreamError::InvalidRecord);

}

// Skip the value

cursor.set_position(end_position);

has_records = true;

}

if !has_records {

return Err(TlvStreamError::EmptyStream);

}

// If validation passes, create the tagged hash

Ok(Self::from_valid_tlv_stream_bytes(tag, bytes))

pub fn from_tlv_stream_bytes(tag: &'static str, bytes: &[u8]) -> Result<Self, TlvStreamError> {

// Validate the TLV stream first

if bytes.is_empty() {

return Err(TlvStreamError::EmptyStream);

}

// Try to parse the TLV stream to check validity

let mut cursor = io::Cursor::new(bytes);

let mut has_records = false;

let mut previous_type: Option<u64> = None;

while cursor.position() < bytes.len() as u64 {

// Try to read type

let type_result = <BigSize as Readable>::read(&mut cursor);

if type_result.is_err() {

return Err(TlvStreamError::InvalidRecord);

}

let current_type = type_result.unwrap().0;

// Check ascending order

if let Some(prev) = previous_type {

if current_type <= prev {

return Err(TlvStreamError::InvalidOrder);

}

}

previous_type = Some(current_type);

// Try to read length

let length_result = <BigSize as Readable>::read(&mut cursor);

if length_result.is_err() {

return Err(TlvStreamError::InvalidRecord);

}

let length = length_result.unwrap().0;

let end_position = cursor.position() + length;

// Check if the record extends beyond the buffer

if end_position > bytes.len() as u64 {

return Err(TlvStreamError::InvalidRecord);

}

// Skip the value

cursor.set_position(end_position);

has_records = true;

}

if !has_records {

return Err(TlvStreamError::EmptyStream);

}

// If validation passes, create the tagged hash

Ok(Self::from_valid_tlv_stream_bytes(tag, bytes))

Spotted by Diamond

Is this helpful? React 👍 or 👎 to let us know.

ldk-reviews-bot · 2025-07-07T10:48:01Z

🔔 1st Reminder

Hey @TheBlueMatt @valentinewallace! This PR has been waiting for your review.
Please take a look when you have a chance. If you're unable to review, please let us know so we can find another reviewer.

ldk-reviews-bot · 2025-07-07T10:48:02Z

🔔 1st Reminder

Hey @TheBlueMatt @valentinewallace! This PR has been waiting for your review.
Please take a look when you have a chance. If you're unable to review, please let us know so we can find another reviewer.

vincenzopalazzo · 2025-07-07T17:18:52Z

cc @jkczyz

- Add an Option<OfferId> field to Bolt12Invoice to track the originating offer. - Compute the offer_id for invoices created from offers by extracting the offer TLV records and hashing them with the correct tag. - Expose a public offer_id() accessor on Bolt12Invoice. - Add tests to ensure the offer_id in the invoice matches the originating Offer, and that refund invoices have no offer_id. - All existing and new tests pass. This enables linking invoices to their originating offers in a robust and spec-compliant way. Signed-off-by: Vincenzo Palazzo <[email protected]>

jkczyz · 2025-07-08T20:32:08Z

lightning/src/offers/invoice.rs

+				let offer_tlv_stream = TlvStream::new(&self.bytes).range(OFFER_TYPES);
+				let experimental_offer_tlv_stream = TlvStream::new(&self.experimental_bytes).range(EXPERIMENTAL_OFFER_TYPES);
+				let combined_tlv_stream = offer_tlv_stream.chain(experimental_offer_tlv_stream);
+				let tagged_hash = TaggedHash::from_tlv_stream("LDK Offer ID", combined_tlv_stream);
+				Some(OfferId(tagged_hash.to_bytes()))


You can reuse OfferId::from_valid_invreq_tlv_stream (or something with a more appropriate name) rather than duplicating that code.

jkczyz · 2025-07-08T20:36:21Z

lightning/src/offers/invoice.rs

+		let secp_ctx = Secp256k1::new();
+		let payment_id = PaymentId([1; 32]);
+
+		// Create an offer


Probably don't need to the comments throughout these tests if they are just repeating what the code does.

jkczyz · 2025-07-08T20:36:53Z

lightning/src/offers/invoice.rs

@@ -967,6 +987,11 @@ impl Bolt12Invoice {
 		self.tagged_hash.as_digest().as_ref().clone()
 	}

+	/// Returns the offer ID if this invoice corresponds to an offer.


Let's link OfferId and Offer.

jkczyz · 2025-07-08T21:11:50Z

lightning/src/offers/merkle.rs

@@ -41,7 +41,7 @@ impl TaggedHash {
 	/// Creates a tagged hash with the given parameters.
 	///
 	/// Panics if `bytes` is not a well-formed TLV stream containing at least one TLV record.
-	pub(super) fn from_valid_tlv_stream_bytes(tag: &'static str, bytes: &[u8]) -> Self {
+	pub fn from_valid_tlv_stream_bytes(tag: &'static str, bytes: &[u8]) -> Self {


Ok implemented it @jkczyz cfd6ad4 let me know what do you think.

Thanks, left some comments.

However, I still think that ldk should expose a way to verify the signature with the public key that is inside the offer, at least till we do not get a better PoP with bolt12

Given the signature is verified when parsing, shouldn't it be sufficient to check that Bolt12Invoice::signing_pubkey matches the one in the offer? The parser is already doing this when calling check_invoice_signing_pubkey, too, FWIW. Not sure I understand why we need to expose these for the user to piece together and call on their own.

At very least, there doesn't seem to be a need to expose TaggedHash::from_valid_tlv_stream_bytes if we can just have a method on Bolt12Invoice give the TaggedHash for manual signature verification.

ldk-reviews-bot requested a review from valentinewallace June 26, 2025 10:40

vincenzopalazzo force-pushed the macros/markle-signature branch from 0ffed61 to be23002 Compare June 26, 2025 10:44

valentinewallace reviewed Jun 27, 2025

View reviewed changes

vincenzopalazzo requested review from TheBlueMatt and valentinewallace July 5, 2025 10:47

graphite-app bot reviewed Jul 5, 2025

View reviewed changes

vincenzopalazzo force-pushed the macros/markle-signature branch from cfd6ad4 to dd046d5 Compare July 8, 2025 20:40

vincenzopalazzo force-pushed the macros/markle-signature branch from dd046d5 to 29e5bd6 Compare July 8, 2025 20:46

jkczyz reviewed Jul 8, 2025

View reviewed changes

offer: make the merkle tree signature public #3892

Are you sure you want to change the base?

offer: make the merkle tree signature public #3892

Conversation

vincenzopalazzo commented Jun 26, 2025

Uh oh!

ldk-reviews-bot commented Jun 26, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

dunxen commented Jun 26, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

codecov bot commented Jun 26, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Codecov Report

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

ldk-reviews-bot commented Jun 27, 2025

Uh oh!

graphite-app bot Jul 5, 2025

Choose a reason for hiding this comment

Uh oh!

ldk-reviews-bot commented Jul 7, 2025

Uh oh!

ldk-reviews-bot commented Jul 7, 2025

Uh oh!

vincenzopalazzo commented Jul 7, 2025

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Uh oh!

ldk-reviews-bot commented Jun 26, 2025 •

edited

Loading

dunxen commented Jun 26, 2025 •

edited

Loading

codecov bot commented Jun 26, 2025 •

edited

Loading