Clean up index README.md, including doc ToC, and merge architecture and northbound documentations. #1470
Conversation
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
mapedraza
commented
Sep 27, 2023
Comment on lines
-258
to
-277
|
|
||
| ### The `TimeInstant` element | ||
|
|
||
| As part of the device to entity mapping process the IoT Agent creates and updates automatically a special timestamp. | ||
| This timestamp is represented as two different properties of the mapped entity:: | ||
|
|
||
| - With NGSI-v2, an attribute metadata named `TimeInstant` (per dynamic attribute mapped, which captures as an ISO8601 | ||
| timestamp when the associated measurement (represented as attribute value) was observed. With NGSI-LD, the Standard | ||
| `observedAt` property-of-a-property is used instead. | ||
|
|
||
| - For NGSI-v2 only, an additional attribute `TimeInstant` is added to the entity which captures as an ISO8601 | ||
| timestamp when the last measurement received from the device was observed. | ||
|
|
||
| If no information about the measurement timestamp is received by the IoT Agent, the arrival time of the measurement will | ||
| be used to generate a `TimeInstant` for both the entity attribute and the attribute metadata. | ||
|
|
||
| Take into account that: | ||
|
|
||
| - the timestamp of different attributes belonging to the same measurement record may not be equal. | ||
| - the arrival time and the measurement timestamp will not be the same in the general case. | ||
| - if `timezone` field is defined as part of the provisioning of the device or group, timestamp fields will be | ||
| generated using it. For instance, if `timezone` is set to `America/Los_Angeles`, a possible timestamp could be | ||
| `2025-08-05T00:35:01.468-07:00`. If `timezone` field is not defined, by default Zulu Time Zone (UTC +0) will be | ||
| used. Following the previous example, timestamp could be `2015-08-05T07:35:01.468Z`. | ||
|
|
||
| E.g.: in the case of a device that can take measurements every hour of both temperature and humidity and sends the data | ||
| once every day, at midnight, the `TimeInstant` reported for each measurement will be the hour when that measurement was | ||
| observed (e.g. 4:00 PM), while all the measurements will have an arrival time around midnight. If no timestamps were | ||
| reported with such measurements, the `TimeInstant` attribute would take those values around midnight. | ||
|
|
||
| This functionality can be turned on and off through the use of the `timestamp` configuration flag (described in the | ||
| configuration), as well as 'timestamp' flag in device or group provision. |
Comment on lines
-291
to
-292
| ### Implementation decisions | ||
|
|
||
| Given the aforementioned requirements, there are some aspects of the implementation that were chosen, and are | ||
| particularly under consideration: | ||
|
|
||
| - Aside from its configuration, the IoT Agent Lib is considered to be stateless. To be precise, the library mantains a | ||
| state (the list of entities/devices whose information the agent can provide) but that state is considered to be | ||
| transient. It's up to the particular implementation of the agent to consider whether it should have a persistent | ||
| storage to hold the device information (so the internal list of devices is read from a DB) or to register the | ||
| devices each time a device sends a measure. To this extent, two flavours of the Device Registry has been provided: a | ||
| transient one (In-memory Registry) and a persistent one (based in MongoDB). | ||
| - The IoT Agent does not care about the origin of the data, its type or structure. The mapping from raw data to the | ||
| entity model, if there is any, is a responsibility of the particular IoT Agent implementation, or of another third | ||
| party library. |
There was a problem hiding this comment.
Deleted, it does not make sense
Comment on lines
94
to
266
| ## Testing | ||
|
|
||
| Contributions to development can be found [here](doc/devel/development.md) - additional contributions are welcome. | ||
|
|
||
| ### Agent Console | ||
|
|
||
| A command-line client to experiment with the library is packed with it. The command-line client can be started using the | ||
| following command: | ||
|
|
||
| ```console | ||
| bin/agentConsole.js | ||
| ``` | ||
|
|
||
| The client offers an API similar to the one offered by the library: it can start and stop an IoT agent, register and | ||
| unregister devices, send measures mimicking the device and receive updates of the device data. Take into account that, | ||
| by default, the console uses the same `config.js` file than the IoT Agent. | ||
|
|
||
| The command-line client creates a console that offers the following options: | ||
|
|
||
| ```text | ||
| stressInit | ||
| Start recording a stress batch. | ||
| stressCommit <delay> <times> <threads> <initTime> | ||
| Executes the recorded batch as many times as requested, with delay (ms) between commands. | ||
| The "threads" parameter indicates how many agents will repeat that same sequence. The "initTime" (ms) | ||
| parameter indicates the mean of the random initial waiting times for each agent. | ||
| exit | ||
| Exit from the command-line. | ||
| start | ||
| Start the IoT Agent | ||
| stop | ||
| Stop the IoT Agent | ||
| register <id> <type> | ||
| Register a new device in the IoT Agent. The attributes to register will be extracted from the | ||
| type configuration | ||
| unregister <id> <type> | ||
| Unregister the selected device | ||
| showConfig | ||
| Show the current configuration file | ||
| config <newConfig> | ||
| Change the configuration file to a new one | ||
| updatevalue <deviceId> <deviceType> <attributes> | ||
| Update a device value in the Context Broker. The attributes should be triads with the following | ||
| format: "name/type/value" sepparated by commas. | ||
| listdevices | ||
| List all the devices that have been registered in this IoT Agent session | ||
| ``` | ||
|
|
||
| ### Agent tester | ||
|
|
||
| #### Command-line testing | ||
|
|
||
| The library also offers a Context Broker and IoT Agent client that can be used to: | ||
|
|
||
| - Simulate operations to the Context Broker used by the IoT Agent, triggering Context Provider forwardings for lazy | ||
| attributes and checking the appropriate values for active ones. | ||
| - Simulate operations to the Device Provisioning API and Configuration API of the IoT Agent. | ||
|
|
||
| The tester can be started with the following command, from the root folder of the project: | ||
|
|
||
| ```console | ||
| bin/iotAgentTester.js | ||
| ``` | ||
|
|
||
| From the command-line, the `help` command can be used to show a description of the currently supported features. These | ||
| are the following: | ||
|
|
||
| ```text | ||
| stressInit | ||
| Start recording a stress batch. | ||
| stressCommit <delay> <times> <threads> <initTime> | ||
| Executes the recorded batch as many times as requested, with delay (ms) between commands. | ||
| The "threads" parameter indicates how many agents will repeat that same sequence. The "initTime" (ms) | ||
| parameter indicates the mean of the random initial waiting times for each agent. | ||
| exit | ||
| Exit from the command-line. | ||
| update <entity> <type> <attributes> | ||
| Update the values of the defined set of attributes, using the following format: name#type=value(|name#type=value)* | ||
| append <entity> <type> <attributes> | ||
| Append a new Entity with the defined set of attributes, using the following format: name:type=value(,name:type=value)* | ||
| query <entity> <type> | ||
| Get all the information on the selected object. | ||
| queryAttr <entity> <type> <attributes> | ||
| Get information on the selected object for the selected attributes. | ||
| discover <entity> <type> | ||
| Get all the context providers for a entity and type. | ||
| configCb <host> <port> <service> <subservice> | ||
| Config a new host and port for the remote Context Broker. | ||
| showConfigCb | ||
| Show the current configuration of the client for the Context Broker. | ||
| configIot <host> <port> <service> <subservice> | ||
| Config a new host and port for the remote IoT Agent. | ||
| showConfigIot | ||
| Show the current configuration of the client for the IoT Agent. | ||
| provision <filename> | ||
| Provision a new device using the Device Provisioning API. The device configuration is | ||
| read from the file specified in the "filename" parameter. | ||
| provisionGroup <template> <data> <type> | ||
| Provision a group of devices with the selected template, taking the information needed to | ||
| fill the template from a CSV with two columns, DEVICE_ID and DEVICE_NAME. The third parameter, type | ||
| will be used to replace the DEVICE_TYPE field in the template. All the devices will be provisioned | ||
| to the same IoT Agent, once the templates have been fulfilled. | ||
| listProvisioned | ||
| List all the provisioned devices in an IoT Agent. | ||
| removeProvisioned <deviceId> | ||
| Remove the selected provisioned device from the IoT Agent, specified by its Device ID. | ||
| addGroup <filename> | ||
| Add a new device group to the specified IoT Agent through the Configuration API. The | ||
| body is taken from the file specified in the "filename" parameter. | ||
| listGroups | ||
| List all the device groups created in the selected IoT Agent for the configured service | ||
| removeGroup <apiKey> <resource> |
There was a problem hiding this comment.
Added to development.md
Comment on lines
-7
to
-31
| Each Device will be mapped as an Entity associated to a Context Provider: the Device ID will be mapped by default to the | ||
| entity ID and the type of the entity will be selected by the IoT Agent in a protocol-dependent way (e.g: with different | ||
| URLs for different types). Both the name and type will be configurable by the user, either by type configuration or with | ||
| the device preprovisioning. | ||
|
|
||
| Each of the measures obtained from the device should be mapped to a different attribute. The name and type of the | ||
| attribute will be configured by the user (globally for all the types in the IoT Agent configuration or in a per device | ||
| basis preprovisioning the devices). Device measures can have three different behaviors: | ||
|
|
||
| - **Active attributes**: are measures that are pushed from the device to the IoT agent. This measure changes will be | ||
| sent to the Context Broker as updateContext requests over the device entity. NGSI queries to the context broker will | ||
| be resolved in the Broker database. | ||
|
|
||
| - **Lazy attributes**: some sensors will be passive, and will wait for the IoT Agent to request for data. For those | ||
| measures, the IoT Agent will register itself in the Context Broker as a Context Provider (for all the lazy measures | ||
| of that device), so if any component asks the Context Broker for the value of that sensor, its request will be | ||
| redirected to the IoT Agent (that behaves as a NGSI10 Context Provider). This operation will be synchronous from the | ||
| customer perspective: the Context Broker won't return a response until the device has returned its response to the | ||
| IoT Agent. | ||
|
|
||
| - **Commands**: in this case, the interaction will begin by setting an attribute in the device's entity, for which the | ||
| IoT Agent will be regitered as CP. The IoT Agent will return an immediate response to the Context Broker, and will | ||
| be held responsible of contacting the device to perform the command itself, updating special `status` and `info` | ||
| attributes in the entity as soon as it has any information of the command progress. | ||
|
|
doc/devel/architecture.md
Outdated
Comment on lines
70
to
93
| #### Configurations and subservices | ||
|
|
||
| Configurations are meant to be a mean of simplifying the device provisioning for groups of very similar devices. | ||
| Considering that different groups of devices may be created in the same subservice that may require different | ||
| configurations, multiple configurations are allowed for each subservice. Considering the key in the association between | ||
| Device and Configuration was the triplet (service, subservice, type), all of these elements are considered mandatory. | ||
|
|
||
| This statement doesn't hold true for older IoT Agents, though. In older versions of the IoT Agents, each device | ||
| configuration was assigned to a particular subservice and just one configuration was allowed per subservice, so the | ||
| relation between a Device and a Configuration didn't need the type to discriminate between Configurations. That's why | ||
| for those agents, type was not a mandatory parameter. | ||
|
|
||
| In order to allow backward-compatibility with those agents, the IoT Agent Library now implement a compatibility mode: | ||
| the **Single Configuration Mode**, that makes the agent behave like the old agents. In this mode: | ||
|
|
||
| - Each Subservice can contain just one Configuration. If a second Configuration is created for a Subservice, an error | ||
| is raised. | ||
|
|
||
| - Each Device provisioned for a Subservice is automatically assigned to the Subservice one Configuration if there is | ||
| any. | ||
|
|
||
| This compatibility has to be set for the whole IoT Agent, and there is no option of having both modes simultaneously | ||
| running. Transitions from one mode to the other should be made with care, and may involve data migration. | ||
|
|
There was a problem hiding this comment.
Deleted. It does not make sense
#1469
mapedraza
commented
Aug 5, 2024
mapedraza
commented
Aug 5, 2024
mapedraza
commented
Aug 5, 2024
mapedraza
commented
Aug 5, 2024
Comment on lines
-156
to
-201
| ##### NGSI-LD `datasetId` support | ||
|
|
||
| Limited support for parsing the NGSI-LD `datasetId` attribute is included within the library. A series of sequential | ||
| commands for a single attribute can be sent as an NGSI-LD PATCH payload as follows: | ||
|
|
||
| ```json | ||
| { | ||
| "lampColor": [ | ||
| { | ||
| "type": "Property", | ||
| "value": { "color": "green", "duration": "55 secs" }, | ||
| "datasetId": "urn:ngsi-ld:Sequence:do-this" | ||
| }, | ||
| { | ||
| "type": "Property", | ||
| "value": { "color": "red", "duration": "10 secs" }, | ||
| "datasetId": "urn:ngsi-ld:Sequence:then-do-this" | ||
| } | ||
| ] | ||
| } | ||
| ``` | ||
|
|
||
| This results in the following sequential array of attribute updates to be sent to the `UpdateHandler` of the IoT Agent | ||
| itself: | ||
|
|
||
| ```json | ||
| [ | ||
| { | ||
| "name": "lampColor", | ||
| "type": "Property", | ||
| "datasetId": "urn:ngsi-ld:Sequence:do-this", | ||
| "metadata": {}, | ||
| "value": { "color": "green", "duration": "55 secs" } | ||
| }, | ||
| { | ||
| "name": "lampColor", | ||
| "type": "Property", | ||
| "datasetId": "urn:ngsi-ld:Sequence:then-do-this", | ||
| "metadata": {}, | ||
| "value": { "color": "red", "duration": "10 secs" } | ||
| } | ||
| ] | ||
| ``` | ||
|
|
||
| The equivalent for NGSI-v2 is not required since `datasetId` syntax is not supported by NGSI-v2. | ||
|
|
There was a problem hiding this comment.
already present in api.md
mapedraza
changed the title
fgalan
reviewed
Aug 5, 2024
Co-authored-by: Fermín Galán Márquez <[email protected]>
fgalan
reviewed
Aug 5, 2024
| not updated by the device. They are updated by the Context Broker, and the IoT Agent will be in charge of | ||
| translating the updateContext request to the proper action in the device. Two additional attributes are created for | ||
| each command: `status` and `info`. For each command, its `name` and `type` must be provided. | ||
| - **`commands`**: Commands are actions that can be invoked in the device. They are entity attributes, but they are not |
There was a problem hiding this comment.
Maybe a link to the recently added section about Command Execution would be useful at this point.
There was a problem hiding this comment.
Added here: fe57c79
fgalan
reviewed
Aug 5, 2024
| Particular IoT Agents _may_ support autoregistration of devices into configurations, if enough information is given from | ||
| the Southbound. | ||
|
|
||
| #### Configurations and subservices |
There was a problem hiding this comment.
This part has gone to some other .md? Or has been deleted as it is not useful or already described?
There was a problem hiding this comment.
As noted here: #1470 (comment)
and described here: #1469
It is not valid any more
mapedraza
commented
Aug 5, 2024
fgalan
reviewed
Aug 5, 2024
Co-authored-by: Fermín Galán Márquez <[email protected]>
fgalan
reviewed
Aug 5, 2024
fgalan
approved these changes
Aug 5, 2024
There was a problem hiding this comment.
LGTM
Passing the ball to @AlvaroVega for additional review and LGTM before merging.
AlvaroVega
reviewed
Aug 6, 2024
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Part of #1329