[FEATURE] Add Data API support with routing metadata headers and pagination

ChangeLog.md

@@ -7,8 +7,23 @@ and this project adheres to [Semantic Versioning](http://semver.org/spec/v2.0.0.
 
 ## [Unreleased]
 
+### Added
+
+- Data API support with dedicated `DataApi` class
+  - `request()` method for single authenticated Data API requests
+  - `request_iter()` method for iterating through paginated responses
+  - `results_iter()` method for iterating through individual results across pages
+  - Automatic routing metadata headers: `X-Learnosity-Consumer`, `X-Learnosity-Action`, `X-Learnosity-SDK`
+- Data API demo added to Rails quickstart application
+- Comprehensive unit and integration tests for Data API functionality
+- Example usage in `examples/simple/data_api_example.rb`
+
+### Fixed
+
+- Ruby 2.6 compatibility in Rails quickstart (commented out `spring` gems that require Ruby 2.7+)
+- Rails 6.1 compatibility with Ruby 2.6 (added `require 'logger'` to `config/boot.rb`)
 - Bumped 3rd party libraries to fix known vulnerabilities in the quick start application
-- Fixed seed data for the api-reports example in the quick start application  
+- Fixed seed data for the api-reports example in the quick start application
 
 ## [v0.3.0] - 2024-07-12
 ### Added

README.md

@@ -98,6 +98,9 @@ For production use, you should install the SDK using the RubyGems package manage
 Let's take a look at a simple example of the SDK in action. In this example, we'll load an assessment into the browser.
 
 ### **Start up your web server and view the standalone assessment example**
+
+**Note:** The Rails quickstart supports Ruby 2.6+. The `spring` and `spring-watcher-listen` gems are commented out in the Gemfile for Ruby 2.6 compatibility. If you're using Ruby 2.7+, you can uncomment these gems for faster development reloading.
+
 To start up your Ruby web server, first find the following folder location under the SDK. Change directory ('cd') to this location on the command line.
 
 ``` bash
@@ -107,6 +110,7 @@ To start up your Ruby web server, first find the following folder location under
 To start, run this command from that folder:
 
 ``` bash
+    bundle install
     rails server
 ```
 
@@ -277,6 +281,9 @@ Take a look at some more in-depth options and tutorials on using Learnosity asse
 ### **SDK reference**
 See a more detailed breakdown of all the SDK features, and examples of how to use more advanced or specialised features on the [SDK reference page](REFERENCE.md).
 
+### **Data API support**
+The SDK now includes comprehensive Data API support with automatic request signing, routing metadata headers, and pagination support. See the [Data API documentation](docs/DataApi.md) for detailed usage examples and API reference.
+
 ### **Additional quick start guides**
 There are more quick start guides, going beyond the initial quick start topic of loading an assessment, these further tutorials show how to set up authoring and analytics:
 

REFERENCE.md

@@ -106,16 +106,67 @@ request = init.generate
 Net::HTTP.post_form URI('https://data.learnosity.com/v1/itembank/items'), request
 ```
 
-### Recursive Queries
+### DataApi Class (Recommended)
 
-tl;dr: not currently implemented
+The SDK now includes a dedicated `DataApi` class that provides a more convenient way to interact with the Data API, including automatic pagination support, routing metadata headers, and simplified request handling.
+
+```ruby
+require 'learnosity/sdk/request/data_api'
+
+# Initialize DataApi
+data_api = Learnosity::Sdk::Request::DataApi.new(
+  consumer_key: 'your_consumer_key',
+  consumer_secret: 'your_consumer_secret',
+  domain: 'yourdomain.com'
+)
+
+security_packet = {
+  'consumer_key' => 'your_consumer_key',
+  'domain' => 'yourdomain.com'
+}
+
+# Make a single request
+response = data_api.request(
+  'https://data.learnosity.com/v1/itembank/items',
+  security_packet,
+  'your_consumer_secret',
+  { 'limit' => 10 },
+  'get'
+)
+
+# Iterate through all pages automatically
+data_api.request_iter(
+  'https://data.learnosity.com/v1/itembank/items',
+  security_packet,
+  'your_consumer_secret',
+  { 'limit' => 100 },
+  'get'
+).each do |page|
+  puts "Page has #{page['data'].length} items"
+end
+
+# Iterate through individual results
+data_api.results_iter(
+  'https://data.learnosity.com/v1/itembank/items',
+  security_packet,
+  'your_consumer_secret',
+  { 'limit' => 100 },
+  'get'
+).each do |item|
+  puts "Item: #{item['reference']}"
+end
+```
+
+See the [Data API documentation](docs/DataApi.md) for more details and examples.
+
+### Recursive Queries (Legacy Approach)
 
 Some requests are paginated to the `limit` passed in the request, or some
 server-side default. Responses to those requests contain a `next` parameter in
 their `meta` property, which can be placed in the next request to access another
 page of data.
 
-For the time being, you can iterate through pages by looping over the
+You can iterate through pages by looping over the
 `Init#new`/`Init#generate`/`Net::HTTP#post_form`, updating the `next` attribute
 in the request.
 
@@ -129,6 +180,8 @@ end
 
 This will `require 'json'` to be able to parse the response.
 
+**Note:** The new `DataApi` class (see above) handles pagination automatically and is the recommended approach.
+
 See `examples/simple/init_data.rb` for an example.
 
 ### Generating UUIDs

docs/quickstart/lrn-sdk-rails/Gemfile

@@ -44,8 +44,10 @@ group :development do
   gem 'web-console', '>= 4.2.0'
   gem 'listen', '~> 3.8'
   # Spring speeds up development by keeping your application running in the background. Read more: https://github.com/rails/spring
-  gem 'spring'
-  gem 'spring-watcher-listen', '~> 2.1.0'
+  # Note: spring-watcher-listen requires Ruby >= 2.7.0, so it's commented out for Ruby 2.6 compatibility
+  # If you're using Ruby 2.7+, you can uncomment these lines for faster development reloading:
+  # gem 'spring'
+  # gem 'spring-watcher-listen', '~> 2.1.0'
 end
 
 # Windows does not include zoneinfo files, so bundle the tzinfo-data gem

docs/quickstart/lrn-sdk-rails/app/controllers/data_api_controller.rb

@@ -0,0 +1,147 @@
+require 'learnosity/sdk/request/data_api'
+require 'json'
+
+class DataApiController < ApplicationController
+  # rubocop:disable Metrics/CyclomaticComplexity
+  # Note: This is a demo/quickstart controller that intentionally demonstrates
+  # three different Data API usage patterns (manual iteration, page iteration,
+  # and results iteration) with comprehensive error handling for educational purposes.
+  def index
+    # Initialize DataApi
+    data_api = Learnosity::Sdk::Request::DataApi.new(
+      consumer_key: Rails.configuration.consumer_key,
+      consumer_secret: Rails.configuration.consumer_secret,
+      domain: 'localhost'
+    )
+
+    # Endpoint and security packet
+    itembank_uri = 'https://data.learnosity.com/latest-lts/itembank/items'
+    security_packet = {
+      'consumer_key' => Rails.configuration.consumer_key,
+      'domain' => 'localhost'
+    }
+
+    # Get SDK version
+    sdk_version = Learnosity::Sdk::VERSION
+
+    # Initialize request metadata
+    @request_metadata = {
+      endpoint: itembank_uri,
+      action: 'get',
+      status_code: nil,
+      headers: {
+        'X-Learnosity-Consumer' => data_api.send(:extract_consumer, security_packet),
+        'X-Learnosity-Action' => data_api.send(:derive_action, itembank_uri, 'get'),
+        'X-Learnosity-SDK' => "Ruby:#{sdk_version}"
+      }
+    }
+
+    # Demo 1: Manual iteration (5 items)
+    @demo1_output = []
+    @demo1_error = nil
+
+    begin
+      data_request = { 'limit' => 1 }
+
+      5.times do |i|
+        result = data_api.request(
+          itembank_uri,
+          security_packet,
+          Rails.configuration.consumer_secret,
+          data_request,
+          'get'
+        )
+
+        # Capture status code from the first request
+        @request_metadata[:status_code] = result.code if i == 0
+
+        response = JSON.parse(result.body)
+
+        if response['data'] && response['data'].length > 0
+          item = response['data'][0]
+          @demo1_output << {
+            number: i + 1,
+            reference: item['reference'] || 'N/A',
+            status: item['status'] || 'N/A'
+          }
+        end
+
+        if response['meta'] && response['meta']['next']
+          data_request = { 'next' => response['meta']['next'] }
+        else
+          break
+        end
+      end
+    rescue => e
+      @demo1_error = e.message
+    end
+
+    # Demo 2: Page iteration (5 pages)
+    @demo2_output = []
+    @demo2_error = nil
+
+    begin
+      data_request = { 'limit' => 1 }
+      page_count = 0
+
+      data_api.request_iter(
+        itembank_uri,
+        security_packet,
+        Rails.configuration.consumer_secret,
+        data_request,
+        'get'
+      ).each do |page|
+        page_count += 1
+        page_data = {
+          page_number: page_count,
+          item_count: page['data'] ? page['data'].length : 0,
+          items: []
+        }
+
+        if page['data']
+          page['data'].each do |item|
+            page_data[:items] << {
+              reference: item['reference'] || 'N/A',
+              status: item['status'] || 'N/A'
+            }
+          end
+        end
+
+        @demo2_output << page_data
+        break if page_count >= 5
+      end
+    rescue => e
+      @demo2_error = e.message
+    end
+
+    # Demo 3: Results iteration (5 items)
+    @demo3_output = []
+    @demo3_error = nil
+
+    begin
+      data_request = { 'limit' => 1 }
+      result_count = 0
+
+      data_api.results_iter(
+        itembank_uri,
+        security_packet,
+        Rails.configuration.consumer_secret,
+        data_request,
+        'get'
+      ).each do |item|
+        result_count += 1
+        @demo3_output << {
+          number: result_count,
+          reference: item['reference'] || 'N/A',
+          status: item['status'] || 'N/A',
+          json: JSON.pretty_generate(item)[0..500]
+        }
+        break if result_count >= 5
+      end
+    rescue => e
+      @demo3_error = e.message
+    end
+  end
+  # rubocop:enable Metrics/CyclomaticComplexity
+end
+