Non-persistent config shadow changes

[AleksandarGT]

I have enabled config shadows in the configuration because we'd like to change the configuration on demand from the cloud directly. However, the changed configuration is not persistent. After changing the config shadow, if I restart the aws device client service the changes are applied but are not saved in the actual configuration file, so on next service restart / device reboot the changes will revert back to the real configuration file.

It's important to note that we have registered the device client as our own service rather than the automatic way through the setup cli because we had problems, I am not sure if this has anything to do with the current issue.

Our own service file:

[Unit]
Description=AWS IoT Device Client
Wants=network-online.target
After=network.target network-online.target

[Service]
User=sens
Type=simple
Environment="CONF_PATH=/home/sens/.aws-iot-device-client/aws-iot-device-client.conf"
ExecStart=/home/sens/aws-iot-core-connection/aws-iot-device-client/build/aws-iot-device-client --config-file $CONF_PATH
StandardOutput=append:/home/sens/aws-iot-core-connection/aws-iot-device-client/logs/system.log
StandardError=append:/home/sens/aws-iot-core-connection/aws-iot-device-client/logs/system-error.log
Restart=always
RestartSec=5

[Install]
WantedBy=multi-user.target

[ig15]

Hi @AleksandarGT , thanks for bringing up the issue. We are looking into it and will get back to you.

[ig15]

Hey @AleksandarGT. Can you build with the changes in this PR to confirm if it fixes the problem you're facing: #490

[AleksandarGT]

The changes to the shadow via aws now reflects to the configuration file.

However, when making a change to the shadow via the cloud, the config changes for the jobs get includeStdOut omitted. This happens even if I change another configuration option not for the jobs.

Before

  "jobs":	{
    "enabled":	true,
    "includeStdOut": true,
    "handler-directory": "/home/sens/.aws-iot-device-client/jobs"
  },

after

  "jobs":	{
    "enabled":	true,
    "handler-directory": "/home/sens/.aws-iot-device-client/jobs"
  },

Also we use fleet provisioning, so when the initial provisioning is complete a file is created .aws-iot-device-client/aws-iot-device-client-runtime.conf

When I make a change to the config shadow via the cloud, the information from there gets copied over to the aws-iot-device-client.conf. The whole 'runtime-config' json wasn't a part of the configuration before the shadow update. I am not sure this is desired behavior.

"fleet-provisioning":	{
	"enabled":	true,
	"template-name":	"temp-dev",
	"template-parameters":	"{\"DeviceId\":\"AM-zwss\",\"ProductId\":\"zws-v1-test\",\"CreatedById\":\"yaws-4ab4-4194-ae9f-3bea0fdbbeb8\"}"
},
"runtime-config":	{
	"completed-fp":	true,
	"cert":	"/home/sens/.aws-iot-device-client/keys/active-certificate.pem.crt",
	"key":	"/home/sens/.aws-iot-device-client/keys/active-private.pem.key",
	"thing-name":	"thg"
},

Also on making a config update via the shadow, the changes will be applied on next execution of the iot device client. Is there a way to trigger the device client to restart itself, or at least stop so the service executes it again? Otherwise we need to trigger a job that will restart the service.

[ig15]

Can you tell us what your exact use case is, for passing the 'includeStdOut' field in the configuration file since its a parameter in a job document and not a field for the configuration file of device client itself?

Coming to the behavior you are experiencing, since device client is meant to update the entire configuration state of the device as part of the config shadow feature to the cloud, therefore updating the local file with the shadow state is expected to override the configuration with what was stored in cloud. And since 'includeStdOut' was never handled and stored in cloud initially, it was overidded when we try to override the local configuration file.
This also explains why you are seeing the 'runtime-config' json as well when the local configuration file is updated with the device state stored in cloud.

Coming to triggering device client restart automatically upon updating config shadow from the cloud: This can be implemented on a use case specific basis but should not be added to our library since often times users don't want the device to restart with new configurations immediately but instead want to control that behavior on their own will via a restart job and we'd like to keep it that way.
Additionally, Shadows can store your device's state information and make it available to your device, AWS IoT services, your custom apps and other AWS services whether the device is online and connected to AWS IoT or not. Automatic restarts could interfere with this particular desired behavior.

[AleksandarGT]

You're correct I was passing 'includeStdOut' in the config file when it should've been in the job document itself. As for the runtime config json it doesn't really interfere with the other configuration options. For device restart it still could be a nice config options that can be added in the future when the device receives a config shadow update to automatically restart. Until then pushing jobs for restart also works.

Not really a part of this issue but I'd like to mention so you can monitor this issue further is that if a device receives more than 1 job request without restarting, the first job will execute fine, but the second one will get stuck during execution. The second job will either timeout if the job is configured for that, or it will be permanently stuck. Restarting the device sometimes doesn't even work because iot device client prevents the device from rebooting normally and a hard power off / power on is required. To fix this we just enforce the device to restart on every job completion so this problem didn't happen after that.

[ig15]

Thanks for your response @AleksandarGT . Appreciate your inputs here. With respect to the latter section of your response, can you provide the steps for reproducing the undesired jobs behavior (preferably in a new issue)?
Additionally, just an update that we'll be taking some time to merge the PR for the config shadow feature changes, in order to incorporate a .conf field to enable/disable device side persistence of the config shadow update, relevant unit tests and documentation updates

[ig15]

The PR adding a new field to trigger Config Shadow persistent updates has been merged. Closing this issue.