api to query arguments, formats

[cjcliffe]

It'd be nice from a GUI perspective (especially for unknown modules) if the device args, formats, frequency names and stream args could be queried. I'm currently considering creating an xml document to ship with the application to try and hold the parameter information for known modules; but it'd be a lot nicer to just be able to query those things at runtime.

Available Kwargs could be returned by a key=param, value=parameter type (strings, args, ranges) for anything that accepts args.

Functions could be added to query an argument by name per type (strings, args or a ranges).

Would be useful to be able to query the defaults as well.

For GUI configuration this makes it simpler to build input fields to match the device properties without having to populate the form properties from another resource.

[guruofquality]

The key/value pair arguments are for flexibility -- although they are definitely close to being magic. I can see how querying them would go a long way towards structured GUI options for each device. Anyway, here is my brain-dump on this, feedback is appreciated:

Set frequency

setFrequency() -- takes tune arguments which are used for specifying tune offsets, or modifying the behaviour of the overall tune algorithm. This isnt complete magic, as the default overall tune algorithm does have some documentation in terms of how it handles keys of the same name as tunable components.

• Example: ArgsInfo getFrequencyArgsInfo(direction, channel) const

Excusing the name, just following the convention, we might solve this by querying a information structure given a specific channel. So we have different available options on a per channel basis. I'm expecting at least RX and TX to have different options, but there could be heterogeneous channels in the same direction -- imagine an FPGA modification.

Setup stream

setupStream() takes a format string and stream arguments. The format string is more or less predictable, but different implementations offer different formats, and there is the possibility of making up arbitrary strings. The stream arguments are completely arbitrary, they are often used to modify buffer allocation and scaling.

std::vector<string> getStreamFormats(direction, channels) const

ArgsInfo getStreamArgsInfo(direction, channels) const

The idea here is to keep the available formats as a simple list of strings. We pass in the desired channels and direction so the code can make a distinction if it needs to. Most implementations will probably ignore the channels. And them same idea for the stream arguments.

Enumeration/factory

Device enumerate and make takes arbitrary device arguments. Except for the "driver" key which is a predictable key/value based on the driver's registration name, and often used as a device filter for enumeration and instantiation. Common uses are also specifying device urls or ip addresses, and unique device identifications such as serials and hardware ids.

Now the interesting thing about querying the device arguments is that these do not take a device instance. Implementing this is more like having another top-level registration function. So we will probably add to the registration API to take another function pointer. Something like:

ArgsInfo my_driver_getInfo()
{

}

static myDriver Register(&my_driver_getInfo, &my_find, &my_make...)

Sensors

Adding to this, sensors could also use a way to query the units, a nice display name, possibly a range. So it might be fitting to handle this at the same time with the new ArgsInfo data structure.

ArgsInfo data structure

Here is my quick mock-up:

struct ArgInfo
{
    std::string key; //identifier (required)
    std::string default; //default value when not specified (required)
    std::string name; //displayable name (required)
    std::string units; //empty or units like db, hz, etc (optional)
    enum Type{BOOL, INT, FLOAT, STRING} type; //(required)
    RangeList ranges; //list of ranges for numeric types (optional)
    std::vector<string> options; //enumerated discrete options set (optional)
}

typedef std::vector<ArgInfo> ArgsInfo;

Calls will return a list of ArgInfo's and not a dictionary to preserve order. This will provide consistent ordering when its used in a GUI.

[cjcliffe]

Looks good to me; I think I can work with that -- the only thing I might add for consideration is a simple way to provide a different set of defaults when accessed through SoapyRemote -- such as providing the remote scalar or format defaults for the device.

[guruofquality]

Yes. SoapyRemote should be querying the remote options like this and appending it to its own "local" options, with the appropriate key renaming. This keeps everything nice and consistent/transparent.

With regards to the remote format; once we can query this sort of information, SoapyRemote should be able to pick the best default for the transport format and scaling. Its a little nuanced per device, so we should find a consistent way for devices to share their native data format and scaling -- probably through the stream info. RTL would be CS8 with scaling 128, BladeRF would be CS16 with scaling 2048, etc.

[cjcliffe]

SoapyRemote choosing the best format is ideal I think; something like querying the hardware bit depth or scaling factor would be straightforward and I'm guessing bit depth is configurable on some devices too -- no need to tinker to get the right parameters on both ends for the best quality / connection speed balance.

Could even add an option to automatically manage the bit depth and format changes when network conditions become poor in order to maintain the signal.

Edit: Side note that SDRPlay is CS16 /w 2048 scaling as well.

[guruofquality]

@cjcliffe I'm going to git tag SoapySDR and the various support modules. And then we can get a little "wild and crazy" with the API additions, particularly this one. Even though it will be mostly additive.

[cjcliffe]

Excellent; I have a configuration dialog mostly ready for displaying/managing soapy device parameters which this will be perfect for.

[guruofquality]

progress so far: https://github.com/pothosware/SoapySDR/tree/arg_info_work

[guruofquality]

Stream Additions

    /*!
     * Query a list of the available stream formats.
     * \param direction the channel direction RX or TX
     * \param channel an available channel on the device
     * \return a list of allowed format strings
     */
    std::vector<std::string> getStreamFormats(const int direction, const size_t channel) const;

    /*!
     * Get the hardware's native stream format for this channel.
     * This is the format used by the underlying transport layer,
     * and the direct buffer access API calls (when available).
     * \param direction the channel direction RX or TX
     * \param channel an available channel on the device
     * \param [out] fullScale the maximum possible value
     * \return the native stream buffer format string
     */
    std::string getNativeStreamFormat(const int direction, const size_t channel, double &fullScale) const;

    /*!
     * Query the argument info description for stream args.
     * \param direction the channel direction RX or TX
     * \param channel an available channel on the device
     * \return a list of argument info structures
     */
    ArgInfoList getStreamArgsInfo(const int direction, const size_t channel) const;

Tuning Additions

    /*!
     * Query the argument info description for tune args.
     * \param direction the channel direction RX or TX
     * \param channel an available channel on the device
     * \return a list of argument info structures
     */
    virtual ArgInfoList getFrequencyArgsInfo(const int direction, const size_t channel) const;

Sensors additions

   /*!
     * Get meta-information about a sensor.
     * Example: displayable name, type, range.
     * \param name the name of an available sensor
     * \return meta-information about a sensor
     */
    virtual ArgInfo getSensorInfo(const std::string &name) const;

    /*!
     * Get meta-information about a channel sensor.
     * Example: displayable name, type, range.
     * \param direction the channel direction RX or TX
     * \param channel an available channel on the device
     * \param name the name of an available sensor
     * \return meta-information about a sensor
     */
    virtual ArgInfo getSensorInfo(const int direction, const size_t channel, const std::string &name) const;

Settings additions

   /*!
     * Describe the allowed keys and values used for settings.
     * \return a list of argument info structures
     */
    ArgInfoList getSettingInfo(void) const;

[guruofquality]

Find/Factory args info

I was looking into creating some sort of args info call for device args -- but I'm a little on the fence about this: I think that most of the "variable" discovery arguments were just misplaced tune or stream arguments. And any other arbitrary key value settings should be supported by the settings API. Ideally, the find function returns arguments that specifically identify a device, and the factory checks these same arguments to construct the device pointer.

Now I can see how the make() args look arbitrary. But if the args come from an enumerate() result, they arent arbitrary. And I think that OK from a GUI perspective to treat the args are opaque (but displayable) results from enumerate. So we really dont have a need for device arguments that are varied by the user for purposes other than constructing the device instance.

Network devices:
One notable exception to this is specifying a network url or address. We cant enumerate what we don't know to look for. First, I want to fix that statement and make enumeration automatic for SoapyRemote: pothosware/SoapyRemote#12 However, there are still cases where broadcast discovery is not an option. What we probably need is a convention to specify a network location. Currently, these two exist: addr=192.168.10.2 and remote=server:12345

Args for command line:
I still want to support variable arguments as a convenience mechanism to pass stream or settings arguments through a command line utility that takes a simple list of key/value pairs. The simple solution here is probably to 1) stash device args and use them for stream args defaults 2) apply device args not used for actual factory construction to the settings api. This way, nothing is lost for the command line apps, and the regular API entry points still get used as intended.

[cjcliffe]

I'm thinking the device arg queries would be primarily for things like custom/hacked modules (like the multitude of custom RTL drivers kicking around) where you might want to turn an experimental/non-standard feature on/off or provide the known parameters for it in a simple way (GUI).

Broadcast discovery would be perfect for me, I'd love to display the local network devices automatically to make setup simple; otherwise I currently allow adding remotes by name and querying them on demand.

Would it make sense to make two new command line args to split device/stream parameters and when current (non-split) args parameter is used just use args for both device and stream? If the known args can be queried separately for device/stream splitting them would be fairly simple..

[guruofquality]

I'm thinking the device arg queries would be primarily for things like custom/hacked modules (like the multitude of custom RTL drivers kicking around) where you might want to turn an experimental/non-standard feature on/off or provide the known parameters for it in a simple way (GUI).

So in this case, what I am hoping for this that any sort of misc option should be handled through the settings api: getSettingInfo(), writeSetting(), and readSetting(). Also this way, rather than a one-time configuration when the device is instantiated, hooks can be toggled on and off. From a GUI perspective, its a very similar presentation, except that the options only apply to instantiated devices.

Broadcast discovery would be perfect for me, I'd love to display the local network devices automatically to make setup simple; otherwise I currently allow adding remotes by name and querying them on demand.

Working on implementing SSDP as we speak. The remote support module will basically multicast the local network, and then call enumerate every discovered server and return the result.

Would it make sense to make two new command line args to split device/stream parameters and when current (non-split) args parameter is used just use args for both device and stream? If the known args can be queried separately for device/stream splitting them would be fairly simple..

I was mostly thinking about existing apps that use SoapySDR devices through their own API wrappers (like UHD or GrOsmoSDR). However, in both cases I can actually stash the device args and apply them at stream construction time (for example pothosware/SoapyOsmo#3)

And you're right, anything actually written against SoapySDR API can just have a separate stream args option or apply the device args to the stream args in the app layer. Basically, there is no reason to change the SoapySDR modules themselves to support this, the app should just handle it however is best.

[cjcliffe]

Right -- settings API should be fine for that, I agree; looking forward to trying out the broadcast feature :)

[guruofquality]

Ok, so this work is merged. So there's going to be a bunch of modules to add new calls to. I will put some issues in the relevant trackers tonight so its not forgotten. But feel free to get started.

Usually when I add calls, the module will still build against older versions of SoapySDR. But this time around, many of the new calls involve the ArgInfo type. So we either have to ifdefs using the compatibility defines here (https://github.com/pothosware/SoapySDR/blob/master/include/SoapySDR/Version.h#L48) or update the version in CMakeLists.txt to read find_package(SoapySDR "0.4.0" NO_MODULE) This is only a concern if someone is trying to build a updated module against a released or prebuilt version of SoapySDR. The ifdef option is uglier but its a little friendlier.

[cjcliffe]

That was quick; I'll take a look at what's involved to get everything up to date. I wouldn't get too worried about supporting legacy at this point -- save that for the major releases and wait for a more ideal solution. -- "0.4.0" NO_MODULE works fine to help prevent issues for now.

[cjcliffe]

I'd recommend updating SoapySDRUtil to display the additional information; otherwise looking good so far and mostly implemented in SoapyRTLSDR and SoapySDRPlay here.

[guruofquality]

probe should show the additional info now

[guruofquality]

Well I think that’s everything. Good work. Closing!