Composable queries: Allow queries as data sources

docs/extending/data-source.md

@@ -22,7 +22,7 @@ $data_source = HttpDataSource::from_array( [
 
 ## HttpDataSource configuration
 
-### **version**: number (required)
+### \_\_version: number (required)
 
 There is no built-in versioning logic, but a version number is required for best practice reasons. Changes to the data source could significantly affect [queries](query.md). Checking the data source version is a sensible defensive practice.
 
@@ -80,43 +80,6 @@ $zipcode_query = HttpQuery::from_array( [
 ] )
 ```
 
-The goal with design was to provide you with flexibility you need to represent any data source.
-
-## HttpDataSource configuration
-
-### **version**: number (required)
-
-There is no built-in versioning logic, but a version number is required for best practice reasons. Changes to the data source could significantly affect [queries](query.md). Checking the data source version is a sensible defensive practice.
-
-### display_name: string (required)
-
-The display name is used in the UI to identify your data source.
-
-### endpoint: string
-
-This is the default endpoint for the data source and can save repeated use in queries. We would suggest putting the root API URL here and then manipulating it as necessary in individual [queries](query.md).
-
-### request_headers: array
-
-Headers will be set according to the properties of the array. When providing authentication credentials, take care to keep them from appearing in code repositories. We strongly recommend using environment variables or other secure means for storage.
-
-## Additional parameters
-
-You can add any additional parameters that are necessary for your data source. In our [Airtable example](https://github.com/Automattic/remote-data-blocks/blob/trunk/example/airtable/events/register.php), you can see that we are setting values for the Airtable `base` and `table`.
-
-Consider adding whatever configuration would be useful to queries. As an example, queries have an `endpoint` property. Our [Zip code example](https://github.com/Automattic/remote-data-blocks/blob/trunk/example/rest-api/zip-code/zip-code.php) sets the endpoint with a function:
-
-```php
-$zipcode_query = HttpQuery::from_array( [
-    'data_source' => $zipcode_data_source,
-    'endpoint' => function ( array $input_variables ) use ( $zipcode_data_source ): string {
-        return $zipcode_data_source->get_endpoint() . $input_variables['zip_code'];
-    },
-])
-```
-
-The goal with design was to provide you with flexibility you need to represent any data source.
-
 ## Custom data sources
 
 The configuration array passed to `from_array` is very flexible, so it's usually not necessary to extend `HttpDataSource`, but you can do so if you need to add custom behavior.

docs/extending/query.md

@@ -66,9 +66,9 @@ This example features a small subset of the customization available for a query;
 
 The `display_name` property defines the query's human-friendly name.
 
-### data_source: HttpDataSourceInterface (required)
+### data_source: HttpDataSourceInterface|HttpQueryInterface (required)
 
-The `data_source` property provides the [data source](./data-source.md) the query uses.
+The `data_source` property provides the [data source](./data-source.md) the query uses. You can also supply another query as the data source, allowing you to compose queries together. This is useful when you want to access subcollections or related data returned by a "parent" query. Query results are always cached in-memory, so there is no performance penalty for composing queries in this way.
 
 ### endpoint: string|callable
 

inc/Config/Query/HttpQuery.php

@@ -3,7 +3,6 @@
 namespace RemoteDataBlocks\Config\Query;
 
 use RemoteDataBlocks\Config\ArraySerializable;
-use RemoteDataBlocks\Config\DataSource\HttpDataSource;
 use RemoteDataBlocks\Config\DataSource\HttpDataSourceInterface;
 use RemoteDataBlocks\Config\QueryRunner\QueryRunner;
 use RemoteDataBlocks\Validation\ConfigSchemas;
@@ -54,9 +53,9 @@ public function get_cache_ttl( array $input_variables ): null|int {
 	/**
 	 * Get the data source associated with this query.
 	 */
-	public function get_data_source(): HttpDataSourceInterface {
+	public function get_data_source(): HttpDataSourceInterface|HttpQueryInterface {
 		if ( is_array( $this->config['data_source'] ) ) {
-			$this->config['data_source'] = HttpDataSource::from_array( $this->config['data_source'] );
+			$this->config['data_source'] = ArraySerializable::from_array( $this->config['data_source'] );
 		}
 
 		return $this->config['data_source'];
@@ -66,7 +65,7 @@ public function get_data_source(): HttpDataSourceInterface {
 	 * Get the HTTP endpoint for the current query execution.
 	 */
 	public function get_endpoint( array $input_variables ): string {
-		return $this->get_or_call_from_config( 'endpoint', $input_variables ) ?? $this->get_data_source()->get_endpoint();
+		return $this->get_or_call_from_config( 'endpoint', $input_variables ) ?? $this->get_data_source()->get_endpoint( $input_variables );
 	}
 
 	/**
@@ -115,7 +114,7 @@ public function get_request_body( array $input_variables ): ?array {
 	 * @param array $input_variables The input variables for this query.
 	 */
 	public function get_request_headers( array $input_variables ): array|WP_Error {
-		return $this->get_or_call_from_config( 'request_headers', $input_variables ) ?? $this->get_data_source()->get_request_headers();
+		return $this->get_or_call_from_config( 'request_headers', $input_variables ) ?? $this->get_data_source()->get_request_headers( $input_variables );
 	}
 
 	/**

inc/Config/Query/HttpQueryInterface.php

@@ -11,7 +11,7 @@
  *
  */
 interface HttpQueryInterface extends QueryInterface {
-	public function get_data_source(): HttpDataSourceInterface;
+	public function get_data_source(): HttpDataSourceInterface|HttpQueryInterface;
 	public function get_cache_ttl( array $input_variables ): null|int;
 	public function get_endpoint( array $input_variables ): string;
 	public function get_request_method(): string;

inc/Config/QueryRunner/QueryRunner.php

@@ -115,6 +115,12 @@ protected function get_request_details( HttpQueryInterface $query, array $input_
 	 * }
 	 */
 	protected function get_raw_response_data( HttpQueryInterface $query, array $input_variables ): array|WP_Error {
+		// If the data source is itself a query, execute it and return the results.
+		$data_source = $query->get_data_source();
+		if ( $data_source instanceof HttpQueryInterface ) {
+			return $data_source->execute( $input_variables );
+		}
+
 		$request_details = $this->get_request_details( $query, $input_variables );
 
 		if ( is_wp_error( $request_details ) ) {
@@ -212,7 +218,7 @@ public function execute( HttpQueryInterface $query, array $input_variables ): ar
 		}
 
 		// Preprocess the response data.
-		$response_data = $this->preprocess_response( $query, $raw_response_data['response_data'], $input_variables );
+		$response_data = $this->preprocess_response( $query, $raw_response_data['response_data'] ?? $raw_response_data, $input_variables );
 
 		// Determine if the response data is expected to be a collection.
 		$output_schema = $query->get_output_schema();
@@ -224,7 +230,7 @@ public function execute( HttpQueryInterface $query, array $input_variables ): ar
 		$parser = new QueryResponseParser();
 		$results = $parser->parse( $response_data, $output_schema );
 		$results = $is_collection ? $results : [ $results ];
-		$metadata = $this->get_response_metadata( $query, $raw_response_data['metadata'], $results );
+		$metadata = $this->get_response_metadata( $query, $raw_response_data['metadata'] ?? [], $results );
 
 		// Pagination schema defines how to extract pagination data from the response.
 		$pagination = null;

inc/Editor/BlockManagement/ConfigRegistry.php

@@ -6,7 +6,7 @@
 
 use RemoteDataBlocks\Logging\LoggerManager;
 use Psr\Log\LoggerInterface;
-use RemoteDataBlocks\Config\Query\HttpQuery;
+use RemoteDataBlocks\Config\ArraySerializable;
 use RemoteDataBlocks\Config\Query\QueryInterface;
 use RemoteDataBlocks\Editor\BlockPatterns\BlockPatterns;
 use RemoteDataBlocks\Validation\ConfigSchemas;
@@ -178,7 +178,7 @@ private static function create_error( string $block_title, string $message ): WP
 
 	private static function inflate_query( array|QueryInterface $config ): QueryInterface {
 		if ( is_array( $config ) ) {
-			return HttpQuery::from_array( $config );
+			return ArraySerializable::from_array( $config );
 		}
 
 		return $config;

inc/Editor/BlockManagement/ConfigStore.php

@@ -7,6 +7,7 @@
 use RemoteDataBlocks\Config\Query\QueryInterface;
 use RemoteDataBlocks\Logging\LoggerManager;
 use Psr\Log\LoggerInterface;
+use RemoteDataBlocks\Config\DataSource\DataSourceInterface;
 
 use function sanitize_title_with_dashes;
 
@@ -82,7 +83,13 @@ public static function get_data_source_type( string $block_name ): ?string {
 			return null;
 		}
 
-		return $query->get_data_source()->get_service_name();
+		$data_source = $query->get_data_source();
+
+		if ( $data_source instanceof DataSourceInterface ) {
+			return $data_source->get_service_name();
+		}
+
+		return null;
 	}
 
 	/**

inc/Validation/ConfigSchemas.php

@@ -2,9 +2,9 @@
 
 namespace RemoteDataBlocks\Validation;
 
-use RemoteDataBlocks\Validation\Types;
 use RemoteDataBlocks\Config\DataSource\HttpDataSourceInterface;
 use RemoteDataBlocks\Config\Query\HttpQueryInterface;
+use RemoteDataBlocks\Validation\Types;
 use RemoteDataBlocks\Config\Query\QueryInterface;
 use RemoteDataBlocks\Config\QueryRunner\QueryRunnerInterface;
 use RemoteDataBlocks\Editor\BlockManagement\ConfigRegistry;
@@ -77,7 +77,7 @@ private static function generate_remote_data_block_config_schema(): array {
 			'render_query' => Types::object( [
 				'query' => Types::one_of(
 					Types::instance_of( QueryInterface::class ),
-					Types::serialized_config_for( HttpQueryInterface::class ),
+					Types::serialized_config_for( QueryInterface::class ),
 				),
 				'loop' => Types::nullable( Types::boolean() ),
 			] ),
@@ -87,7 +87,7 @@ private static function generate_remote_data_block_config_schema(): array {
 						'display_name' => Types::nullable( Types::string() ),
 						'query' => Types::one_of(
 							Types::instance_of( QueryInterface::class ),
-							Types::serialized_config_for( HttpQueryInterface::class ),
+							Types::serialized_config_for( QueryInterface::class ),
 						),
 						'type' => Types::enum(
 							ConfigRegistry::LIST_QUERY_KEY,
@@ -157,7 +157,9 @@ private static function generate_http_query_config_schema(): array {
 			'cache_ttl' => Types::nullable( Types::one_of( Types::callable(), Types::integer(), Types::null() ) ),
 			'data_source' => Types::one_of(
 				Types::instance_of( HttpDataSourceInterface::class ),
+				Types::instance_of( HttpQueryInterface::class ),
 				Types::serialized_config_for( HttpDataSourceInterface::class ),
+				Types::serialized_config_for( HttpQueryInterface::class ),
 			),
 			'endpoint' => Types::nullable( Types::one_of( Types::callable(), Types::url() ) ),
 			'image_url' => Types::nullable( Types::image_url() ),

tests/inc/Config/QueryTest.php

@@ -5,6 +5,8 @@
 use PHPUnit\Framework\TestCase;
 use RemoteDataBlocks\Config\Query\HttpQuery;
 use RemoteDataBlocks\Tests\Mocks\MockDataSource;
+use RemoteDataBlocks\Tests\Mocks\MockQuery;
+use RemoteDataBlocks\Tests\Mocks\MockQueryRunner;
 
 class QueryTest extends TestCase {
 	private MockDataSource $data_source;
@@ -77,4 +79,24 @@ public function testCustomPreprocessResponse(): void {
 
 		$this->assertSame( $expected_json, $custom_query_context->preprocess_response( $html_data, [] ) );
 	}
+
+	public function testQueryAsDataSource(): void {
+		$mock_qr = new MockQueryRunner();
+		$mock_qr->addResult( 'foo', 'bar' );
+
+		$query_with_query_as_data_source = HttpQuery::from_array( [
+			'data_source' => MockQuery::create( [ 'query_runner' => $mock_qr ] ),
+			'output_schema' => [
+				'type' => [
+					'nested_foo' => [
+						'path' => '$.results[0].result.foo.value',
+						'type' => 'string',
+					],
+				],
+			],
+		] );
+
+		$result = $query_with_query_as_data_source->execute( [] )['results'][0]['result']['nested_foo'];
+		$this->assertSame( 'bar', $result['value'] );
+	}
 }

tests/inc/Validation/ValidatorTest.php

@@ -672,6 +672,48 @@ public function testSerializedConfigForSubclass(): void {
 		$this->assertSame( 'Object must have valid property: extra_value', $result->get_error_data()['child']->get_error_message() );
 	}
 
+	public function testOneOfSerializedConfig(): void {
+		$schema = Types::object( [
+			'config' => Types::one_of(
+				Types::serialized_config_for( MockSerializableClass::class ),
+				Types::serialized_config_for( MockSerializableSubclass::class )
+			),
+		] );
+
+		$validator = new Validator( $schema );
+
+		$this->assertTrue( $validator->validate( [
+			'config' => [
+				'__class' => MockSerializableSubclass::class,
+				'boolean_value' => true,
+				'enum_value' => 'foo',
+				'string_value' => 'hello, world!',
+				'extra_value' => 'required for subclass',
+			],
+		] ) );
+
+		$this->assertTrue( $validator->validate( [
+			'config' => [
+				'__class' => MockSerializableClass::class,
+				'boolean_value' => true,
+				'enum_value' => 'foo',
+				'string_value' => 'hello, world!',
+			],
+		] ) );
+
+		$result = $validator->validate( [
+			'config' => [
+				'boolean_value' => true,
+				'enum_value' => 'foo',
+				'string_value' => 'hello, world!',
+			],
+		] );
+
+		$this->assertInstanceOf( WP_Error::class, $result );
+		$this->assertSame( 'Object must have valid property: config', $result->get_error_message() );
+		$this->assertSame( 'Value must be one of the specified types: {"boolean_value":true,"enum_value":"foo","string_value":"hello, world!"}', $result->get_error_data()['child']->get_error_message() );
+	}
+
 	public function testStringMatching(): void {
 		$schema = Types::string_matching( '/^foo$/' );
 

tests/integration/RDBTestCase.php

@@ -97,7 +97,7 @@ protected function register_remote_data_block_from_block_title( string $block_ti
 	}
 
 	protected function get_query_runner_with_response( array $response_data, int $status_code = 200 ): QueryRunner {
-		return new class($response_data, $status_code) extends QueryRunner {
+		return new class( $response_data, $status_code ) extends QueryRunner {
 			private $response_data;
 			private $status_code;
 
@@ -139,6 +139,13 @@ protected function get_dom_element_by_html_id( DOMDocument $dom, string $html_id
 		return $nodes;
 	}
 
+	protected function get_dom_elements_by_html_class( DOMDocument $dom, string $html_class ): DOMNodeList|false {
+		$xpath = new DOMXPath( $dom );
+		$nodes = $xpath->query( sprintf( "//*[@class='%s']", $html_class ) );
+
+		return $nodes;
+	}
+
 	protected function assertDomIdHasTextContent( DOMDocument $dom, string $html_id, string $expected_content ): void {
 		$id_nodes = $this->get_dom_element_by_html_id( $dom, $html_id );