Add offline cache support with SCITOKENS_KEYCACHE_FILE environment variable and enhanced scitokens-keycache tool #172

Copilot · 2025-08-13T20:36:14Z

This PR implements offline support for the scitokens library, enabling environments where external network access to fetch public keys is not available or desired.

Key Features Added

1. SCITOKENS_KEYCACHE_FILE Environment Variable

Added support for a new environment variable that points directly to a SQLite database file for the key cache, bypassing the normal cache location resolution:

export SCITOKENS_KEYCACHE_FILE=/path/to/offline.db

The cache location priority is now:

SCITOKENS_KEYCACHE_FILE (highest priority - for offline use)
User-configured cache home via config API
XDG_CACHE_HOME environment variable
~/.cache directory (lowest priority)

2. Enhanced scitokens-keycache Command Line Tool

Completely redesigned the command-line utility with subcommands for comprehensive cache management:

`scitokens-keycache add`

Creates and populates offline cache files with JWKS data:

scitokens-keycache add --cache-file /opt/tokens/cache.db \
                       --jwks issuer_keys.json \
                       --issuer https://tokens.example.com \
                       --valid-for 86400

`scitokens-keycache print`

Displays a formatted table of all cached public keys with truncated values for terminal readability:

scitokens-keycache print [--cache-file /path/to/cache.db]

# Output shows: Issuer | Key ID | Key Type | Expires | Next Update | Public Key (truncated)

`scitokens-keycache location`

Shows the active cache file location and accessibility status:

scitokens-keycache location

# Output:
# Cache file location: /home/user/.cache/scitokens/scitokens_cpp.sqllite
# Access status: Success - cache file can be read

3. Enhanced API

Added a new C API function for programmatic storage of JWKS with explicit expiration times:

int keycache_set_jwks_with_expiry(const char *issuer, const char *jwks, 
                                  int64_t expires_at, char **err_msg);

// New function for accessing cache location
const char* scitokens_get_cache_file_location();

Implementation Details

Modified get_cache_file() in scitokens_cache.cpp to check for the environment variable first
Extended the internal Validator class with store_jwks_with_expiry() method
Completely restructured scitokens-keycache.cpp with subcommand architecture
Added proper C header includes for int64_t type compatibility
Enhanced table formatting with terminal-friendly truncation for readability

Backward Compatibility

All existing functionality remains unchanged. The new features are additive:

Existing APIs continue to work without modification
Default cache behavior is preserved when environment variable is not set
Legacy scitokens-keycache usage patterns still work via the add subcommand
No breaking changes to the public interface

Testing

Added comprehensive functionality testing for all subcommands
Verified environment variable integration across all commands
Tested error handling for invalid inputs, missing files, and access issues
Confirmed existing test suite continues to pass, ensuring no regressions
Validated table formatting with various JWKS configurations

Use Cases

This enables fully offline scitokens operation and enhanced cache management for:

Air-gapped systems without internet access
High-security environments with restricted network access
Administrative workflows requiring cache inspection and debugging
Batch processing systems that need predictable key availability
Disaster recovery scenarios where external services may be unavailable

The enhanced tool provides administrators with comprehensive visibility into cached keys and simplified workflows for offline deployments.

💡 You can make Copilot smarter by setting up custom instructions, customizing its development environment and configuring Model Context Protocol (MCP) servers. Learn more Copilot coding agent tips in the docs.

…he tool Co-authored-by: djw8605 <[email protected]>

…lity Co-authored-by: djw8605 <[email protected]>

djw8605 · 2025-08-14T16:11:33Z

@copilot Modify the scitokens-keycache file to have different options.

the command line: scitokens-keycache add will allow adding of JWKS to a keycache file, like it is implemented now.
the command line: scitokens-keycache print will print out a table of all public keys stored in the cache. Truncate the public key values to fit a regular sized terminal. Print out the truncated public key, key id, issuer, expires, and next update values.
the command line: scitokens-keycache location prints out the location of the scitokens keycache, given the environment variable and rules already in the get_cache_file function. Also, it will print out if it is successfully able to read the cache file returned by get_cache_file.

…cation) Co-authored-by: djw8605 <[email protected]>

Copilot · 2025-08-14T16:27:37Z

@copilot Modify the scitokens-keycache file to have different options.

the command line: scitokens-keycache add will allow adding of JWKS to a keycache file, like it is implemented now.
...

I've successfully modified the scitokens-keycache tool to support the requested subcommands:

scitokens-keycache add - Adds JWKS to a keycache file (same functionality as before)
scitokens-keycache print - Prints a formatted table showing:

Issuer (truncated to fit terminal)
Key ID
Key Type (RSA, EC, etc.)
Expires date/time
Next Update date/time
Public Key material (truncated for display)

scitokens-keycache location - Shows the scitokens keycache file location based on environment variables and configuration rules, plus indicates if the cache file can be successfully read.

All commands work with both explicit --cache-file arguments and the SCITOKENS_KEYCACHE_FILE environment variable. The implementation maintains full backward compatibility while adding the new functionality.

Commit: 8bdaa4a

Warning

Firewall rules blocked me from connecting to one or more addresses

I tried to connect to the following addresses, but was blocked by firewall rules:

esm.ubuntu.com
- Triggering command: /usr/lib/apt/methods/https (dns block)

If you need me to access, download, or install something from one of these locations, you can either:

Configure Actions setup steps to set up my environment, which run before the firewall is enabled
Add the appropriate URLs or hosts to the custom allowlist in this repository's Copilot coding agent settings (admins only)

… enhance documentation

…nternal.h

github-actions

Remaining comments which cannot be posted as a review comment to avoid GitHub Rate Limit

lint

[lint] _{reported by reviewdog 🐶}

	std::cout << std::setw(40) << "Issuer"
	<< std::setw(15) << "Key ID"
	<< std::setw(15) << "Key Type"
	<< std::setw(20) << "Expires"
	<< std::setw(20) << "Next Update"
	<< std::setw(25) << "Public Key (truncated)" << "\n";

	<< std::setw(15) << "N/A"
	<< std::setw(15) << "N/A"
	<< std::setw(20) << "Invalid JSON"
	<< std::setw(20) << "N/A"
	<< std::setw(25) << "N/A" << "\n";

	if (expires_iter != top_obj.end() && expires_iter->second.is<int64_t>()) {
	time_t expires_time = static_cast<time_t>(expires_iter->second.get<int64_t>());

	if (next_update_iter != top_obj.end() && next_update_iter->second.is<int64_t>()) {
	time_t next_update_time = static_cast<time_t>(next_update_iter->second.get<int64_t>());


	if (keys_array_iter != jwks_obj.end() && keys_array_iter->second.is<picojson::array>()) {
	auto keys_array = keys_array_iter->second.get<picojson::array>();

	std::cout << std::setw(40) << truncate_string(issuer_str, 37)
	<< std::setw(15) << "N/A"
	<< std::setw(15) << "No keys"
	<< std::setw(20) << expires_str
	<< std::setw(20) << next_update_str
	<< std::setw(25) << "N/A" << "\n";

	if (n_iter != key_obj.end() && n_iter->second.is<std::string>()) {
	public_key_snippet = n_iter->second.get<std::string>();
	} else if (x_iter != key_obj.end() && x_iter->second.is<std::string>()) {
	public_key_snippet = x_iter->second.get<std::string>();


	std::cout << std::setw(40) << (first_key ? truncate_string(issuer_str, 37) : "")
	<< std::setw(15) << truncate_string(kid, 12)
	<< std::setw(15) << kty
	<< std::setw(20) << (first_key ? expires_str : "")
	<< std::setw(20) << (first_key ? next_update_str : "")
	<< std::setw(25) << truncate_string(public_key_snippet, 22) << "\n";

	std::cout << std::setw(40) << truncate_string(issuer_str, 37)
	<< std::setw(15) << "N/A"
	<< std::setw(15) << "No keys array"
	<< std::setw(20) << expires_str
	<< std::setw(20) << next_update_str
	<< std::setw(25) << "N/A" << "\n";

	<< std::setw(15) << "N/A"
	<< std::setw(15) << "No JWKS"
	<< std::setw(20) << expires_str
	<< std::setw(20) << next_update_str
	<< std::setw(25) << "N/A" << "\n";

	if (!key_val.is<picojson::object>()) continue;

	const char* cache_location = scitokens_get_cache_file_location();


	auto rv = keycache_set_jwks_with_expiry("https://offline.test.com",
	demo_scitokens2.c_str(),
	expires_at, &err_msg);

+              #include <cstdlib>
+              #include <cstring>
+              #include <ctime>
+              #include <iostream>

+              #include <string>
+              #include <unistd.h>
+              #include <iomanip>
+              #include <sqlite3.h>

+                  std::cout << "Commands:\n";
+                  std::cout << "  add       Add JWKS to a keycache file\n";
+                  std::cout << "  print     Print table of all public keys stored in cache\n";
+                  std::cout << "  location  Print location of scitokens keycache and access status\n";

	std::cout << " location Print location of scitokens keycache and access status\n";
	std::cout << " location Print location of scitokens keycache and access "
	"status\n";

		std::cout << "Expires at: " << ctime(&now) << " + " << valid_for << " seconds\n";

-        std::cout << "Expires at: " << ctime(&now) << " + " << valid_for << " seconds\n";
+        std::cout << "Expires at: " << ctime(&now) << " + " << valid_for
+                  << " seconds\n";

+                      if (err_msg) {
+                          free(err_msg);
+                      }

+                      std::cerr << "Error: " << e.what() << "\n";
+                      return 1;
+                  }

Add offline cache support with SCITOKENS_KEYCACHE_FILE environment variable and enhanced scitokens-keycache tool #172

Are you sure you want to change the base?

Add offline cache support with SCITOKENS_KEYCACHE_FILE environment variable and enhanced scitokens-keycache tool #172

Uh oh!

Conversation

Copilot AI commented Aug 13, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Key Features Added

1. SCITOKENS_KEYCACHE_FILE Environment Variable

2. Enhanced scitokens-keycache Command Line Tool

scitokens-keycache add

scitokens-keycache print

scitokens-keycache location

3. Enhanced API

Implementation Details

Backward Compatibility

Testing

Use Cases

Uh oh!

djw8605 commented Aug 14, 2025

Uh oh!

Copilot AI commented Aug 14, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

I tried to connect to the following addresses, but was blocked by firewall rules:

Uh oh!

github-actions bot left a comment

Choose a reason for hiding this comment

Uh oh!

github-actions bot Aug 14, 2025

Choose a reason for hiding this comment

Uh oh!

github-actions bot Aug 14, 2025

Choose a reason for hiding this comment

Uh oh!

github-actions bot Aug 14, 2025

Choose a reason for hiding this comment

Uh oh!

github-actions bot Aug 14, 2025

Choose a reason for hiding this comment

Uh oh!

github-actions bot Aug 14, 2025

Choose a reason for hiding this comment

Uh oh!

github-actions bot Aug 14, 2025

Choose a reason for hiding this comment

Uh oh!

github-actions bot Aug 14, 2025

Choose a reason for hiding this comment

Uh oh!

github-actions bot Aug 14, 2025

Choose a reason for hiding this comment

Uh oh!

github-actions bot Aug 14, 2025

Choose a reason for hiding this comment

Uh oh!

github-actions bot Aug 14, 2025

Choose a reason for hiding this comment

Uh oh!

github-actions bot left a comment

Choose a reason for hiding this comment

Uh oh!

github-actions bot Aug 14, 2025

Choose a reason for hiding this comment

Uh oh!

github-actions bot Aug 14, 2025

Choose a reason for hiding this comment

Uh oh!

github-actions bot Aug 14, 2025

Choose a reason for hiding this comment

Uh oh!

github-actions bot Aug 14, 2025

Choose a reason for hiding this comment

Uh oh!

github-actions bot Aug 14, 2025

Choose a reason for hiding this comment

Uh oh!

github-actions bot Aug 14, 2025

Choose a reason for hiding this comment

Uh oh!

github-actions bot Aug 14, 2025

Copilot AI commented Aug 13, 2025 •

edited

Loading

`scitokens-keycache add`

`scitokens-keycache print`

`scitokens-keycache location`

Copilot AI commented Aug 14, 2025 •

edited

Loading


		int print_command(int argc, char *argv[]) {
		std::string cache_file;

+                  std::string cache_file;
+                  // Parse command line arguments for print command
+                  for (int i = 2; i < argc; i++) {  // Start from 2 since argv[1] is "print"

	for (int i = 2; i < argc; i++) { // Start from 2 since argv[1] is "print"
	for (int i = 2; i < argc; i++) { // Start from 2 since argv[1] is "print"