Enterprise tenant

Overview

An Enterprise tenant offers additional administrative functionality compared to a Standard tenant, the major difference being multi-tenancy.

Using an Enterprise tenant, you can:

• Create and manage subtenants.
• Manage the subscribed applications/features of the subtenants.
• Invoice subtenants based on usage statistics.

Moreover, an Enterprise tenant includes the following additional features:

• Branding - to configure an individual look & feel.
• Domain name - to provide an individual domain name.
• User hierarchy - to reflect entities with limited permissions to subsets of shared data.

For details on the Cloud of Things tenant concept see Tenant hierarchy in the Concepts guide.

The following sections describe the following additional functionalities available in the Enterprise tenant:

Managing tenants

Using the Enterprise tenant of Cloud of Things, you can make use of the tenants functionality which allows you to create and manage subtenants.

To be able to use the tenant functionality, your user needs to have the appropriate permissions. See To add a global role for information on editing permissions. Since editing tenants is a sensitive operation, permissions for editing tenants are more granular:

• READ - browse and view tenants.
• CREATE - create new tenants.
• UPDATE - edit tenants (incl. subscriptions) and suspend or activate them.
• CHANGE - create, edit and delete tenants.

To view subtenants

Click Subtenants in the Tenants menu to view all subtenants available in your account.

The Tenants page provides the following information on each subtenant:

• The name of the subtenant, for example, company name of your customer.
• The ID and domain.
• Optionally, a contact name.
• The date when the tenant was created.
• The status of the tenant, either active (indicated by a green checkmark icon) or suspended (indicated by a red cross icon).

To create a subtenant

1. Click Create tenant at the right of the top menu bar.

   

2. Provide the following properties:

   Field	Description	Required
   Domain/ URL	Enter a subdomain of your choice, for example "acme". The tenant's URL will be "acme.ram.m2m.telekom.com" on ram.m2m.telekom.com. You can only use one subdomain level. For example, you can only use "acme.ram.m2m.telekom.com" on ram.m2m.telekom.com. You cannot use "mycustomer.acme.ram.m2m.telekom.com". This is not permitted by the TLS standard. The tenant domain may contain lowercase letters, digits or hyphens (-). It must start with a letter; hyphens are only allowed in the middle; minimum is 2 characters.	Yes
   Name	The name of the tenant, for example, the company's name.	Yes
   Administrator's email	A valid email address to enable users to reset their password.	Yes
   Administrator's username	Username for the administrator of this tenant.	Yes
   Contact name	Name of the contact.	No
   Contact phone	Phone number of the contact.	Yes
   Send password reset link as email	Selected by default. If you deselect this option, you must provide a password and confirm the password (see Getting Started > Accessing and logging into the platform > How to log into the platform for more information on password strength).	No
   Tenant policy	You may select a tenant policy to be applied to the tenant from the dropdown list.	No

3. Click Save to apply your settings.

When the subtenant is created, it gets an auto-generated ID, which cannot be changed. Also, it is automatically provisioned with a first, administrative user (“Administrator’s username”). This administrator can create other users and set their permissions. The first user cannot be deleted to prevent you from locking yourself out.

From the Management tenant, you can enable other tenants to create subtenants. To do so, check Allow creation of subtenants in the tenant editor.

To view or edit subtenant properties

Click on the desired subtenant or click the menu icon at the right of the subtenant entry and then click Edit.

In the Properties tab, all fields are editable except of ID, Domain/ URL and Administrator’s username. For details on the fields, refer to To create a subtenant.

To change the tenant password, click Change password, enter the new password in the upcoming fields and click Save.

Support user access information

At the right of the Properties tab, you can find information on the support user requests/access for the subtenants.

The following information is displayed here:

Suspending subtenants

Suspending a tenant blocks any access to this tenant, regardless whether the access is from devices, users or other applications. In addition all its microservices are undeployed, and if the tenant is reactivated all its microservices are re-deployed.

The tenant’s data remains in the database and can be made available later by clicking Activate.

Refer to Usage statistics and billing > Lifecycle for details on the billing perspective of suspended tenants.

To suspend a subtenant

1. Click the menu icon at the right of the respective subtenant entry and then click Suspend.

   

2. In the resulting dialog box confirm the suspension by clicking Suspend and then entering your password.

Deleting subtenants

To delete a subtenant

Click the menu icon at the right of the respective subtenant entry and then click Delete to finally delete a tenant and remove all the data of the tenant.

Applications

In the Applications tab you can view all subscribed applications, subscribe tenants to applications or remove the applications from the tenant. By default, tenants will be subscribed to the standard Cloud of Things applications.

To subscribe an application

Hover over the application under Available applications at the right and click Subscribe on the desired application.

To unsubscribe an application

Hover over the application under Subscribed applications at the left and click Unsubscribe.

Monitoring microservices

For all applications hosted as microservices by Cloud of Things the status of the microservice is indicated next to its name by symbols and may be in one of the following states:

• Microservice is up and running
•   Microservice is unhealthy
• Microservice is down

You may view details on their status by expanding the respective entry.

The following information is provided:

• Active - the number of active microservice instances
• Unhealthy - the number of inactive microservice instances
• Desired - the number of desired microservice instances
• Name - microservice instance name
• Restarts - the number of microservice instance restarts.

Further details are provided on the Status tab of the respective application, see Administration > Managing applications.

Custom properties

The Custom properties tab allows you to view and edit values of custom properties, either predefined ones (like “External reference”) or those defined in the Properties library. Such properties are also displayed as columns in the Usage statistics page.

Limiting subtenant request rate

Platform administrators can limit the request rate of each subtenant via the following custom properties:

• Limit HTTP queue - limit of HTTP request queue for tenant
• Limit HTTP requests - limit of HTTP requests for tenant per second
• Limit stream queue - limit of MQTT request queue for tenant
• Limit stream requests - limit of MQTT requests for tenant per second

The request throttling mechanism is only enabled when both HTTP properties (limit HTTP queue and limit HTTP requests) are configured. If one of the values is omitted, it does not work.

It is also possible to customize the buffer size for the CEP queue and the data broker queue for a particular tenant. This can be done from the Management tenant by using the following subtenant custom fragments:

• cep.queue.limit
• data-broker.queue.limit

When there is no limit on tenant and system level, the limit feature is considered as disabled and the tenant gains unlimited access. To switch off request rate limiting after it was enabled, set the value to “-1”.

Limiting subtenant device number

Platform administrators can limit the count of concurrently registered root devices or simply all devices (including child devices) via the custom property “Limit number of devices”.

They can view the peak number of concurrently registered devices, root devices and the peak value of used storage in the Usage statistics page.

Product experience tracking

Using the checkbox Enable Gainsight product experience tracking a parent tenant can enable/disable the product experience tracking through the Gainsight PX product experience software for the given child tenant.

On tenant level, the product experience tracking by Gainsight can be disabling by disabling the cookie banner on the Branding page, see Customizing your platform > Branding.

Even if tracking is enabled for a tenant, users must actively accept the tracking of functional cookies, before any functional data on the usage of the platform is tracked, see Getting started > Accessing and logging into the platform.

Tenant policies

A tenant policy is a set of tenant options and retention rules. Tenant options and retention rules may be specified during tenant creation.

Creating a tenant policy with a specific set of options and rules saves time when creating multiple tenants with the same settings.

To view tenant policies

Click Tenant policies in the Tenants menu to view all available tenant policies.

For each tenant policy, the name, an optional description and the number of options and retention rules is provided, either in a list or a grid.

To create a tenant policy

1. Click Add policy in the top menu bar.
2. In the resulting dialog box, enter a name and an optional description.
   
   
3. Add at least one retention rule. For details on creating retention rules, see Administration > Managing data > Retention rules.
4. Optionally, add a tenant option.
5. Click Save.

The tenant policy will be added to the tenant policies list.

To edit a tenant policy

Click the respective policy entry or click the menu icon at the right of the policy entry and then click Edit.

In the resulting dialog box, make your edits and click Save to save your settings.

To delete a retention rule or a tenant option from a policy, hover over it and click the delete icon.

To duplicate a tenant policy

Click the menu icon in the tenant policy entry you want to duplicate and then click Duplicate.

To delete a tenant policy

Click the menu icon in the tenant policy entry you want to delete and then click Delete.

Default subscriptions

In the Cloud of Things platform, you can configure which applications and microservices are subscribed to a tenant on tenant creation. When you create a new tenant, the specified applications and microservices automatically get subscribed to it.

In addition, you can specify which applications and microservices are subscribed to a tenant when the system is upgraded. This list might differ from the default subscriptions on tenant creation. For example, certain default applications might have been unsubscribed from a tenant after creation and you may not want these applications to be subscribed to it again or you may want to subscribe different ones to it.

In the Default subscriptions page, you can configure two separate lists of applications. These will be subscribed by default to:

• Every new tenant on its creation.
• Every existing tenant on platform upgrade.

On the left, the list of subscribable applications (both web applications and microservices) is displayed, which consists of:

• All own applications.
• All subscribed applications which have different names than the own applications.

On the right, you see the Subscribed on tenant creation and the Subscribed on platform upgrade columns.

Initially, the lists show the default subscriptions inherited from the tenant hierarchy.

You can override both lists by switching the corresponding toggle. This will reveal all available applications (initially, unselected ones are hidden) but the selection will remain the same.

Next, adjust the lists to your needs by selecting additional applications to be subscribed by default or deselect applications you do not want to be subscribed.

You may also deselect all of them if you don’t want any subscriptions to be executed on tenant creation and/or platform upgrade.

If you want to return to the settings inherited from the tenant hierarchy, just switch the corresponding toggle again.

Save the settings by clicking Save at the bottom of the page.

Overriding default subscriptions

The default subscriptions can be overridden for subtenants by setting up a tenant policy with the following options:

• To define default web applications subscribed to new tenants on creation:
  • category: configuration
  • key: default.tenant.applications
  • value: comma-separated list of application names, for example, administration,devicemanagement,cockpit,feature-microservice-hosting,feature-cep-custom-rules
• To define default microservices subscribed to new tenants on creation:
  • category: configuration
  • key: default.tenant.microservices
  • value: comma-separated list of microservice names, for example, device-simulator,report-agent,sms-gateway
• To use a different list of web applications to be subscribed to existing tenants on platform upgrade:
  • category: configuration
  • key: on-update.tenant.applications.enabled
  • value: true/false (when false or not set, the same list from default.tenant.applications will be used)
• To define default web applications subscribed to existing tenants on platform upgrade:
  • category: configuration
  • key: on-update.tenant.applications
  • value: comma-separated list of application names, for example, administration,devicemanagement,cockpit,feature-microservice-hosting,feature-cep-custom-rules
• To use a different list of microservices to be subscribed to existing tenants on platform upgrade:
  • category: configuration
  • key: on-update.tenant.microservices.enabled
  • value: true/false (when false or not set, the same list from default.tenant.microservices will be used)
• To define default microservices subscribed to existing tenants on platform upgrade:
  • category: configuration
  • key: on-update.tenant.microservices
  • value: comma-separated list of microservice names, for example, device-simulator,report-agent,sms-gateway

Managing user hierarchies

With user hierarchies you can reflect independent organizational entities in Cloud of Things that still share the same database. These entities can have limited permissions to subsets of the shared data and can manage their own sub-users.

Viewing user hierarchies

In the Users page, user hierarchies are indicated by an arrow left from the user icon. Clicking on the arrow unfolds the user hierarchy. You can also fold and unfold the entire user hierarchy using the Expand all and Collapse all links at the right of the top menu bar.

A small number next to the user name shows how many direct sub-users a user has. Sub-users are users that can be managed by their respective parent user and that have at most the permissions of that parent user. In the example below, the user “Demo user” has one direct sub-user.

To create a sub-user

User hierarchies are created by assigning an “owner” to a user. The owner can manage the user. The user can have at most the same permissions as the owner.

1. Select the user in the Users page.
2. In the Owner field, select the user you want to assign as owner from the dropdown list.
3. Click Done to confirm.

Example

A user A has the role “business”. User A becomes the owner of a new user B. User B can then only get a business role assigned (and not for example an admin role) as the user cannot have higher permissions than the owner. If you try to assign any other role except “business” for user B, then the role will be unavailable for subscription and will be indicated by a warning icon with a notification that this operation is not permitted.

Delegating user hierarchies to other users

In Cloud of Things, users can delegate their user hierarchies and permissions to another user. The delegated user then has the same user management permissions as the user who activated the delegation. To do user management, the delegated user must have CREATE permission for the “User management” permission type, which can be granted by assigning a predefined global role “Shared User Manager” or by assigning a custom global role with this permission.

You may of course also delegate on a temporary basis, for example if you are temporarily unavailable.

To delegate permissions to a user

Either open the user and click the delegate icon in the Delegated by field, or click the menu icon at the right of the user entry in the user list and from the context menu, select Delegate.

To undelegate permissions

Remove the delegation in the Delegate by field, or click the menu icon at the right of the user entry in the user list and from the context menu, select Undelegate.

If the delegated user also needs to manage specific devices, the admin user must assign this device permissions (inventory roles) directly to the intended user. This can be done by using Copy inventory roles from another user. For details refer to Administration > Managing permissions > Assigning inventory roles to users.

Troubleshooting sub-users

In the example below the user cannot change the access to the Administration application, because the owner of the user has no “User management” permission. As a result, the owner user can not assign built-in applications (and the owned user cannot use them).

Customizing your platform

With the Enterprise tenant of Cloud of Things, you can customize your platform in various aspects and according to your requirements.

Apart from various configuration settings, you can use your individual branding and your individual domain name.

Click Enterprise tenant in the Settings menu to access these settings.

Configuration

The following placeholders can be found in the Configuration tab:

Two-factor authentication

Under Two-factor authentication, you can change the SMS template which is sent to the users.

Support link

In the Support link section, you can enter a URL to be used to link to a support page. If you do not provide a link here, the default link to the Tech Community page will be used.

Enter “false” to hide the link.

Password reset

In the Password reset section you can change all settings related to password reset email templates.

At the top you can select if you want to allow sending emails to unknown email addresses.

In the Password reset email template fields, provide an email template to be used when the address is known and one to be used when the address is unknown. The link to reset the password might for example be: {tenant-domain}/apps/devicemanagement/index.html?token={token}&email={email}.

In the Email subject field, provide a subject for all password reset related emails.

In the following two fields provide an email template to be used on password change confirmation and a template for the invitation email.

Email server

In the Email server section, you can configure custom email server settings.

In the Protocol and encryption field, select a protocol/encryption type from the dropdown list. May be one of:

• SMTP (no encryption): email.protocol=smtp and email.connection.encrypted=false
• SMTP (STARTTLS): email.protocol=smtp and email.connection.encrypted=true
• SMTPS (SSL/TLS): email.protocol=smtps and email.connection.encrypted=true

Provide the host, port, username, password, and sender address for the email server. The empty password configuration is supported for the Enterprise tenant.

Data export

In the Data export section, you can set the email subject and email template for data export and specify the User unauthorized error message.

Storage limit

In the Storage limit section, you can specify the email subject and email template for emails being send before data is removed on exceeding the storage limit (warning) and after data removal is performed (limit exceeded).

Suspending tenants

In the Suspending tenants section, you can provide settings for emails being send on tenant suspension.

At the top you can select if you want to send the email to the suspended tenant’s administrator and specify an additional email receiver. Below you set the subject and template for the tenant suspended email.

Click Save configuration at the bottom to save your settings.

Branding

With the Branding feature, you can fully customize the look of your tenant to your own preferences.

In the Branding tab, you can configure various parameters like logos, colors and font types used throughout the platform.

The parameters are configured at the left side of the tab while at the right you can immediately see your selections applied to a preview extract.

For a more detailed preview of your settings, click Open preview in the top menu bar to check the look and feel of your branding settings in the overall platform. You may interact and even switch applications in the preview. Every change that you make in the Branding tab will immediately be applied to the Preview page.

When you are done or want to store your settings, click Save at the bottom of the Configuration section to save your branding settings to your tenant.

Saving the settings will not yet apply them to the current tenant and respective subtenants. To do so, click Apply in the top menu bar.

Click Remove branding in the top menu bar to reset the branding of the current tenant and its subtenants to the default settings. The custom settings will still be saved but are no longer applied.

Configuration parameters

In the Configuration section, the following branding parameters can be configured.

General

Under General, you can edit the title which will be used in the browser tab.

Main logo

Under Main logo, specify the following items:

• The favicon - will be displayed in the browser’s address bar. Click Choose file to select a file from your file system. The supported favicon format is “ico”.
• Your branding logo - will be shown during application loading. Click Choose file to select a file from your file system. The supported formats are “png”, “svg” and “jpg”.
• The brand logo height.

Navigator logo

Under Navigator logo you can provide the navigator logo and set the navigator logo height located on top of the navigator panel.

Font

In the Font section you specify the font settings for your branded version.

You can select your base and headings font stack, and select an option for the navigator font stack (either same as base or same as headings font). You may also add a link to existing remote fonts to be used.

Colors

In the Colors section you specify the colors to be used in your branding version.

The following parameters can be specified by providing a hex, rgb or rgba value:

• Main brand color.
• Secondary brand color - the default value is “#07b91A”.
• Dark brand color - mainly used for two-color icons. The default value is “#0B385B”.
• Light brand color - mainly used for two-color icons. The default value is “#5FAEEC”.
• Text color - the default value is “#444”.
• Link color - the default value is the same as the main brand color.
• Main background color - the default value for this item is “#FAFAFA”.

Top bar

In the Top bar section you specify the parameters for the top bar.

The following parameters can be specified by providing a hex, rgb or rgba value:

• Background color - the default value is “#FFFFFF”.
• Text color - the default value is “49595B”.
• Button hover text color - the default value is the main brand color.

Navigator

In the Navigator section you specify the parameters for the navigator.

The following parameters can be specified by providing a hex, rgb or rgba value:

• Background color - the default value is “#2c3637”.
• Logo wrapper background color - the default value is “Transparent”.
• Title color - the default value is “#FFFFFF”.
• Text and buttons color - the default value is “#FAFAFA”.
• Separator line color - the default value is “#FAFAFA”.
• Text color of the current item in the navigator - the default value is “#FAFAFA”.
• Background color of the current item in the navigator with the main brand color as default.

Misc

In the Misc section you specify the “Button Border-Radius” by providing a value in pixel (px).

Cookie banner

In the Cookie banner section you specify the settings for the banner with the cookie usage information. If not disabled here, the banner is shown for all users of the current tenant and all subtenants until a user clicks Agree and proceed.

Disabling the cookie banner, also disables the product experience tracking by Gainsight for the current tenant and all subtenants.

The following parameters can be specified:

• Title - cookie banner title
• Text - cookie banner text with a general statement on the cookie usage and the use cases for it
• Link to privacy policy - a link to the page with the privacy policy

Domain name

A key feature of the Enterprise tenant is the ability to operate the Cloud of Things platform using a custom domain name. This means that you can configure the platform to serve you and your customers using a host name of choice, for example *.iot.mycompany.com rather than the default URL of Cloud of Things. In addition you’ll be able to create subtenants using your domain. These will be using <subtenantName>.iot.mycompany.com as their host names.

There are three prerequisites for using a custom domain:

1. To activate your domain, a valid license that covers your wildcard domain is required. Please contact product support to install a license for your domain.
2. You’ve obtained a valid wildcard SSL certificate for your IoT domain, for example a certificate for *.iot.mycompany.com.
3. There is a valid DNS configuration for your domain which ensures that all requests to *.iot.mycompany.com are routed to Cloud of Things. (see below).

SSL certificate requirements

The following criteria must be met by any SSL certificate to be used with the Enterprise tenant feature:

• The certificate is currently valid and has not expired. More specifically, validFrom points to a point in time in the past, and validTo to a point in the future.
• The certificate has been issued by a well-established certificate authority (CA). Self-signed certificates are explicitly not supported.
• The certificate is a wildcard certificate issued for your domain *.iot.mycompany.com. The use of a wildcard certificate is mandatory, as it will also be used for subdomains created from your Enterprise tenant.
• Every single certificate in the chain is provided using the X509 format.
• The common name (CN) in the subject of the primary certificate (the first one in the chain) holds the value of your wildcard domain name, for example, “CN=*.iot.mycompany.com”.

Cloud of Things supports a single certificate that is signed by the root CA, as well as a full chain certificate which contains one or more intermediate certificates.

Packaging the SSL certificate in PKCS #12

In order to use an SSL certificate with Cloud of Things, the certificate together with its private key must be uploaded to the platform in a single file, using the PKCS #12 file format.

Most certificate authorities deliver their certificates and corresponding private keys in the PEM file format, using two separate text files for the certificate chain and the private key. Make sure that the private key is not protected with a password/passphrase.

Such PEM files can easily be repackaged into #PKCS #12 using OpenSSL. In the following example, OpenSSL is used to combine a certificate chain (chain.cert) and the corresponding key (privkey.pem) into a PKCS #12 keystore file (out_keystore.p12) that can be used with Cloud of Things.

openssl pkcs12 -export -out out_keystore.p12 -inkey privkey.pem -in cert.pem -certfile chain.pem

DNS requirements for enterprise domains

The DNS entries for your custom domain must be configured in a way that all requests are routed to the Cloud of Things platform.

We strongly recommend you to use a wildcard CNAME entry for this purpose. The CNAME needs to contain your wildcard domain from the certificate in the NAME field. The VALUE field of the CNAME entry must point to the hostname of Cloud of Things. This target hostname can be easily determined by looking at your current tenant URL. If your tenant URL is http://mytenant.ram.m2m.telekom.com, the target hostname is mytenant.ram.m2m.telekom.com. Please also make sure to delete any conflicting A entries.

Example:

If you want to use *.iot.mycompany.com for your enterprise subtenants and if you’re using the Cloud of Things at mytenant.ram.m2m.telekom.com, the following CNAME entry must be added to your DNS zone:

NAME                   TYPE   VALUE
----------------------------------------------------
*.iot.mycompany.com.   CNAME  mytenant.ram.m2m.telekom.com.

We highly discourage any use of alternative DNS configurations for the following reasons:

• Wildcard A entries take the IP address of the platform in the value field and hence redirect all requests based on the given IP rather than a hostname. This results in major problems if the IP address of the IoT platform should change in the future.
• Singular A entries or singular CNAME entries instead of DNS wild cards require a single DNS entry for each enterprise domain being created. This is very error prone and prevents the creation of subtenants without always tampering with DNS settings.

Uploading the certificate and activating your domain

Once the DNS configuration is in place and if a certificate with the given requirements is available, it can be easily uploaded to the platform.

On the Domain name tab in the Enterprise page, click Upload certificate. Select the certificate from your file system and click Upload.

Afterwards, you can activate the domain with a single click on its name. After the domain has been activated, you will be redirected to your Enterprise tenant using the new domain name. You will also receive an email with information about the activation. Note that your Management tenant domain name is static, for example, if your wildcard domain is “* .iot.mycompany.com” then your Management tenant domain will be “management.iot.mycompany.com”.

Updating your certificate

When your certificate expires, you must update your certificate with a new one with an extended validation period. When updating a certificate, you must make sure that the certificate meets the following requirements:

• It is valid, like when being uploaded for the first time.
• It is currently valid (validFrom in the past and validTo in the future).
• It has exactly the same common name (domain name) as the currently active certificate.

Deactivating your certificate

If you wish to return to your old domain at Cloud of Things, you can simply deactivate you certificate.

Troubleshooting

In case you cannot reach Cloud of Things using your custom domain, we recommend you to perform the following checks to verify your DNS setup.

Check if the DNS entry is correct

Execute the following command:

host management.<your domain name>

The following result should be returned:

management.<your domain name> is an alias for <instance domain name>
<instance domain name> has address <ip address>

Check if the API is responding

Execute the following command:

curl -v -u '<tenant ID>/<your user>:<your password>' --head http://management.<your domain name>/inventory/managedObjects

The following result should be returned:

...
HTTP/1.1 200 OK
...

Support user access

The support user access feature enables Cloud of Things platform providers (DT IoT in case of the public cloud instances, or service providers in case of individual on-prem installations) to support their customers by accessing their users using a support user. A support user is a user in the Management tenant that has specific permissions, that is, to access subtenant users in case of any issues.

To use this feature, support user access must be configured and the required support users must be created in the Management tenant, see Administration > Platform configuration settings > Support user.

Configuring support user access

Support user access can either be:

• Activated for all subtenants by default.
• Deactivated for all subtenants, but explicitly be enabled by a user for their tenant.

This is configured globally in the Management tenant, see Administration > Platform configuration settings > Support user.

If activated globally, the support user can log in to all allowed subtenants as any user without restriction.

If deactivated globally, support user access can still be enabled by a subtenant user if required. This is done by clicking Enable support in the User menu, see Getting started > User options and settings. The support access is not restricted to the user who activated it but applies to all users of the tenant. This is necessary for retracing of role/right issues.

After a user has activated support access, the menu item changes to Disable support, so that the user can disable a pending support request which has been resolved actively before it expires.

The duration of the active support request can be globally configured in the Management tenant (default is 24 hours), see Administration > Platform configuration settings > Support user.

Each new support request will prolong the support duration for the specified number of hours. After the last support request in a subtenant has expired or has been actively disabled by the user, the support user access for the subtenant will immediately be disabled (if not activated globally).

Details on the status of support requests and support user access for a tenant can be found in the Properties tab of the tenant, see Managing tenants.

To log in as support user

To log in as support user from the Management tenant, you must provide the following information in the Login screen:

• Tenant ID - the ID of the tenant of the user to be supported. The tenant ID is shown in the user menu in the UI.
• Support user - the username of the Management tenant user that executes the support.
• User - the username of the user to be supported.

“Support user” and “user” are entered into the Username field in the following notation:

<support user>$<user>

Example

Suppose you get a support call from a user “John” in the tenant testtenant.ram.m2m.telekom.com (which has the tenant ID t07007007). Your username in the Management tenant is “Jill” and you are permitted to carry out support for testtenant.ram.m2m.telekom.com. In this case, you can log in with the following credentials to reproduce what John is seeing:

Alternatively, enter “<support user>$” into the Username field to access the tenant with one of the administrative users.

Audit logs are created for each support user access and for the actions that support users perform. In the column “Who?” the author’s name will be shown in the form of “support_user$user”.

License management

The Management tenant may manage the licenses for the domains of the Enterprise tenants via the UI.

To add and validate a domain license

1. Log into the Management tenant.
2. Navigate to the License management page under the Settings menu. In the License management page all domain licenses are listed, together with their status (green checkmark = valid).
3. Paste the license key into the Add tenant license field and click Add.

The license will be validated and added to the tenants options.

To delete a domain license

1. Log into the Management tenant.
2. Navigate to the License management page under the Settings menu.
3. Click the menu icon at the right if the domain for which you want to delete the license and then click Delete.

Usage statistics and billing

Viewing usage statistics

The Usage statistics page provides statistical information on each subtenant.

The following information is provided for each subtenant (not completely visible in the screenshot above due to space restrictions):

Moreover custom properties are displayed, if configured.

Custom properties may be defined in the Properties library and then set their values in the Custom properties tab of the tenant.

You can filter the usage statistics list for a time period by adding the start and end date in the top menu bar and click Apply. The Usage statistics page will show the numbers for all subtenants for this time period.

You can also filter and sort the list on any column by clicking the filter icon next to the column name and providing the filtering criteria. See also Getting Started > UI functionalities and features > Filtering.

To export the usage statistics table

1. Click Export CSV at the right of the top menu bar to export the current view of the statistics table to a CSV file.
2. In the resulting dialog box you can customize the CSV output by specifying a field separator, decimal separator and charset.
3. Click Download to start the export.

The CSV file will be downloaded to your file system.

Devices count details

The device count calculations assume that only top-level devices are marked with the device marker fragment c8y_IsDevice. Accordingly, the following formulas are used in the calculations:

• root devices - all devices with the c8y_IsDevice fragment
• all devices - all devices with the c8y_IsDevice fragment and their children from the whole device hierarchy
• leaf devices - only leafs of the device hierarchies starting from devices with the c8y_IsDevice fragment

If child devices are also marked with the c8y_IsDevice fragment, the calculation results may look different than expected.

Microservice usage

The microservice usage feature gathers information on the resource usage per subtenant for each microservice. This enables Enterprise tenants and service providers to charge tenants not only based on subscriptions but also based on resources usage.

Billing modes

Cloud of Things offers two billing modes:

• Subscription-based billing - charges a constant price when a tenant is subscribed to a microservice while resource usage is assigned to the owner.

• Resource-based billing - exposes the number of resources used by a microservice to calculate billing.

The billing modes are specified per microservice in the microservice manifest and are set in the field “billingMode”.

RESOURCES: Sets the billing mode to resources-based. This is the default mode and will be applied to all microservices that are not explicitly switched to subscription-based billing mode.

SUBSCRIPTION: Sets the billing mode to subscription-based.

Isolation level

Two isolation levels are distinguished for microservices: per-tenant isolation and multi-tenant isolation.

In case of subscription-based billing, the entire resources usage is always assigned to the microservice owner, independent of the isolation level, while the subscribed tenant will be billed for the subscription.

In case of resources-based billing, charging depends on the isolation level:

• Per-tenant - the subscriber tenant is charged for used resources.
• Multi-tenant - the owner of the microservice is charged for used resources.

In case of multi-tenant isolation level, the owner of a microservice (for example the Management tenant of an Management tenant or service provider) is charged for the used resources of the subtenants. The subtenants should be charged based on the subscription according to the agreement between the microservice owner and the subscribed tenant. The list of subscribed applications is available as part of the tenant applications as subscribedApplications.

Resources usage assignment for billing mode and isolation level

Collected values

The following values are collected on a daily base for each tenant:

• CPU usage, specified in CPU millicores (1000m = 1 CPU)
• Memory usage, specified in MB

Microservice resources are counted based at limits defined in the microservice manifest per day. At the end of each day, the information about resource usage is collected into the tenant statistics. It is also considered that a microservice might not be subscribed for a whole day.

Example: If a tenant was subscribed to a microservice for 12h and the microservice has 4 CPU and 4 GB of memory it should be counted as 2000 CPU millicores and 2048 MB of memory.

For billing purposes, in addition to CPU usage and memory usage the cause for the billing is collected (for example owner, subscription for tenant):

{
  "name": "cep",
	"cpu": 6000,
	"memory": "20000",
	"cause": "Owner"
},
{
  "name": "cep-small",
  "cpu": 1000,
  "memory": "2000",
  "cause": "Subscription for tenant"
}

The information on the microservice usage is presented in the Usage statistics page.

For more details, refer to Tenants in the Cloud of Things OpenAPI Specification. Note that details are available only for daily usage. For a summary query only the sum of all issued requests is returned.

Scaling

Auto-scaling monitors your microservices and automatically adjusts capacity to maintain steady, predictable performance at the lowest possible cost. It is easy to configure the microservice scaling by setting the property scale in the Microservice manifest.

For instance, when you have a microservice with scale policy set to AUTO and the CPU usage points that it is needed to start a new microservice instance for three hours, the billing logs: (24/24 + 3/24) * consumed resources.

• 24/24 - one instance active for the whole day
• 3/24 - second instance active only three hours

Note that an audit record is created for every change of the number of instances.

For more information, refer to Audits in the Cloud of Things OpenAPI Specification.

Timezone handling

The tenant usage statistics are collected on a daily base according to the beginning of day (BOD) and the end of day (EOD), which are defined by the server timezone. As a result, if the local time zone of a user is different from the server timezone, an operation triggered by the user may be assigned to a different day according to the server time.

Examples

Request counting - Example 1

Result:

The request will be billed to the day 25.08.2020 as this is the server time of the request handing.

Request counting - Example 2

Result:

The request will be billed to the day 26.08.2020 as the server time is the same as the device time.

Microservice resource billing - Example 1

Result:

The resources will be assigned mainly to the day 26.08.2020 as according to the UTC time zone the microservice was active for 14 hours that day and for 10 hours the next day. This might be a bit different from what a user expects as from his perspective the microservice was active for 12 hours each day.

Microservice resource billing - Example 2

Result:

From the user perspective the microservice was subscribed for 8 hours at 26.08.2020 but at server time it was 2 hours before EOD of 25.08.2020 and 6 hours after BOD at 26.08.2020.

Microservice resource billing - Example 3

Result:

In this case we have a big time shift between the server and the user time. All resources will be billed to the day 25.08.2020 according to the server time.

Daily routine

Usage statistics consist of values that are progressive like the request count and values that are snapshots of a state at a given time period. In case of the second type of data, values are refreshed several times each day but the value from EOD is the value that is assigned for the given day.

Lifecycle

Tenant

A Cloud of Things platform tenant can have several states:

• Active - the common state when the tenant can interact with the platform. In that state all billing values are stored and updated.
• Suspended - suspended tenants are not billed for request count and microservice resources, the only value that is still counted is the existence of the tenant and the storage size. The microservice resource usage is billed as “used”, that means, when the tenant is switched to suspended state all microservices are stopped so there are no resources to bill.
• Deleted - this is the point of no return. The tenant is not billed for any resources but there is no way of restoring the data also.

Microservice

Any extension deployed to the platform as a microservice is billed as “used” and the billing starts according to the begin of usage. After the application is subscribed to the tenant a process of application startup is triggered which will go through several high level phases:

• Pending - the microservice has been scheduled to be started but the Docker container is not running yet. In this state the microservice is not yet billed.
• Scheduled - the microservice has been assigned to a node, the Docker container initialization has been started. The resources for the microservice have already been allocated so billing is started.
• Not ready - the microservice container is not ready yet to handle incoming traffic but the application is already running.
• Ready - the microservice container is ready to handle incoming traffic. “Ready” is resolved based on liveness and readiness probes defined in the microservice manifest. If probes are not defined then the microservice is immediately ready.

A tenant that is billed for resources can view the point in time when the microservices billing has been changed in the audit logs. The audit log entries, for example “Scaling application ‘…’ from X to Y instances” contain the information about the changes of instances and resources consumed by the microservice.

Tenants should also be able to see the full application lifecyle in the application details. In the Status tab, you can see an Events section that is showing very low level stages of the application startup. Some of the most important are:

• Pod "apama-ctrl-starter-scope-..." created. - a new microservice instance has been scheduled to be started for the tenant. This means that the resource allocation has been successful but the application is not running yet (maps to the state “Scheduled”).
• Pulling image "apama-ctrl-starter-scope-..." - the microservice initialization process has been started and the Docker image download is already in progress (state “Scheduled”).
• Container created. - the microservice container has been created but not started yet (state “Scheduled”).
• Container started. - the microservice container is started but not ready yet to handle incoming traffic (state “Not ready”).

Audit logs and events are stored at tenant space according to the isolation level. For multi-tenant isolated microservices this is the tenant that is the owner of the microservice and in case of per-tenant isolation level it is the subscribed tenant.

Billing pricing models

The Cloud of Things platform collects a lot of different usage statistics data which is used for billing customers.

Based on the contract, there are two pricing models for billing:

• Tenant usage pricing model - based on tenant usage statistics
• Device pricing model - based mostly on device statistics and microservice resource usage

The table below presents which values are used in each model for billing purposes:

Using the data broker

Data broker lets you share data selectively with other tenants. You can share:

• Devices (and more generically, managed objects)
• Events
• Alarms
• Measurements
• Operations

Navigate to Data connectors in the Data broker menu if you would like to send data to another tenant. Navigate to Data subscriptions, if you would like to receive data from another tenant.

Be aware of the following limitations of the data broker:

• Remote Control cannot be used on the destination tenant.
• The Management tenant cannot be used as data broker source tenant.
• Currently, the Fieldbus widget does not work on tenants that receive the fieldbus devices through data broker, as the corresponding data models are not synchronized.
• Data broker does not guarantee the same order of messages on destination tenants as it was on the source tenant.
• While we provide backwards compatibility, we cannot ensure that data broker can send data to Cloud of Things tenants which run on earlier Cloud of Things versions than the source.

Data connectors

A data connector describes the subset of the data that you would like to send to a destination tenant as well as the URL of that destination tenant.

Viewing data connectors

Click Data connectors in the navigator to see a list of all currently defined data connectors with their status.

For each data connector, the following information is provided:

• The data connector’s name
• Its destination tenant
• Description if provided, “None” displayed otherwise
• The status of the connector
• The number of filters set for the data connector

Use the toggle to enable and disable data forwarding to the destination tenant. If data is being forwarded, the toggle reads “Active”. If data is not being forwarded, the toggle reads “Suspended” or “Pending”. “Suspended” means that you have disabled forwarding. “Pending” means that the destination tenant has disabled forwarding.

To add a data connector

1. Click Add data connector in the top menu bar.

2. In the Settings tab, provide the following information to create a new data connector:

   Field	Description
   Title	The name of the data connector.
   Target URL for data connector	The URL of the tenant to which data will be forwarded. Once saved, you cannot edit this value anymore.
   Description	A textual description of the configuration. Both the name and the description will be visible on the destination side after accepting the subscription.
   Data filters	A set of filters that define what is copied to the destination. You must configure at least one filter.

3. Click Add filter to configure a new filter.

   

4. Each data filter contains the following information:

   Field	Description
   Group or device	The group or device that is forwarded. If you select a group here all subgroups and subdevices of this group will be forwarded. See the warning below on the usage of All objects.
   API	The type of data being forwarded (alarms, events, measurements, managed objects) or being received (operations).
   Fragments to filter	The fragments that must be present in a device to be forwarded.
   Fragments to copy	The fragments that are copied to the destination. If nothing is specified here, only standard properties of managed objects, alarms, events and measurements are forwarded (see below). Select Copy all fragments to forward the entire object.
   Type filter	Forwarded data needs to have this value in its "type" property.

5. Click Save to save the configuration.

If the Group or device field is filled in, the entire descendant structure of the inventory is forwarded to the destination as soon as the connector stays active. if the Group or device field is empty or set to “all” the descendant structure of the inventory is not forwarded; in this case the filter works in “lazy” mode, meaning it forwards the device or asset along with its first event/measurement/alarm.

If operation API is checked in filters, operations created in the target tenant will be forwarded to the source tenant. This applies only to operations that meet the following conditions:

• The operation’s device itself is a result of forwarding data.
• The operation matches other filter criteria.

Updates of the operation status coming from the source tenant will be forwarded to the destination tenant.

The heading of a data filter summarizes the configuration in one line. The standard properties that are copied by default are:

• For created alarms: type, text, time, severity, status
• For updated alarms: status, text, severity
• For created events: type, text, time
• For created measurements: type, text, time
• For created and updated devices: type, name, c8y_IsBinary, c8y_IsDeviceGroup, c8y_IsDevice, c8y_DeviceGroup, c8y_DeviceSubgroup, c8y_SmartRule, c8y_DynamicGroup, c8y_DeviceQueryString
• For updated operations: status

After saving the configuration, you will see a security code displayed below your configuration. The security code prevents unintended forwarding of data. You must communicate this security key separately to an administrative user of the destination tenant. You can click the copy icon next to the security code to copy the code to your clipboard.

To edit a data connector

Click the connector title, or click the menu icon at the right of a data connector entry and then click Edit.

In the Settings tab, edit the data connector configuration.

See To add a data connector for details on the settings.

To duplicate a data connector

Click the menu icon at the right of a data connector entry and then click Duplicate to create another data connector with the same configuration.

To delete a data connector

Click the menu icon at the right of a data connector entry and then click Delete to stop data forwarding and delete the data connector.

To view alarms for a data connector

Open a data connector and switch to the Alarms tab to display current alarms for the data connector.

For details on data broker alarms, see Troubleshooting below.

For details on alarms in general, see Device management > Monitoring and controlling devices > Working with alarms.

Data subscriptions

In the Data subscriptions page, you can manage existing data subscriptions or create new ones.

Click Data subscriptions to see a list of all currently defined data forwarded to your tenant.

For each subscription, the name, the target tenant and the status (enabled or disabled) is provided on a card.

Use the toggle to temporarily stop forwarding data into your tenant.

To set up data forwarding on the receiving end

1. Click Add data subscription in the top menu bar to receive data.
2. In the new card, enter the security code that you received from the sending end of the data.
3. When the connection is established, click Accept to start forwarding data into your tenant. The subscription is active now.
4. You can use the toggle in the card to temporarily stop forwarding data into your tenant.

You can now navigate to the Device management application or the Cockpit application. You will find a new “virtual group” with a specific icon (see the screenshot below) showing the forwarded devices. The group will have the same name as your subscription. Devices are “lazily” created on the destination side whenever they send data for the first time after setting up an active subscription.

To delete a data connector

Click the menu icon and then click Delete to stop data forwarding and delete the data connector.

Troubleshooting

If the data broker is not able to connect to a destination tenant, a CRITICAL alarm is raised, showing the connector which is affected.

Queue overflow

On the source tenant, data broker queues data that cannot be forwarded immediately to the destination tenant. The amount of data that can be queued is limited. If Cloud of Things cannot queue any further data, the oldest queued data is dropped. In this case, a MAJOR alarm is raised in the tenant, showing the connector which is affected.

Similarly, an alarm is raised when the input queue is overflown.

To reduce the number of alarms, alarms are not triggered more often than once per minute.

Microservice-based data broker

The microservice-based data broker is powered by the Cloud of Things Messaging Service that enables reliable, scalable and high-performance movement of IoT data. The microservice-based data broker is similar to the existing data broker in its functionality, except that a microservice, the databroker-agent-server, must be enabled to make use of it.

To enable the microservice-based data broker

1. In the Management tenant, navigate to Administration > Tenants > Subtenants, and select the tenant that will serve as the source of the data connector.
2. In the Applications tab:
   • Subscribe the tenant to the data broker feature (feature-broker application), if it is not already subscribed.
   • Subscribe the tenant to the data broker agent (databroker-agent-server application).

Data connectors

See Data connectors for details on how to manage data connectors.

Data subscriptions

See Data subscriptions for details on how to manage data subscriptions.

Migrating existing data connectors to the microservice-based data broker

After enabling the microservice-based data broker, your existing data connectors should continue to work without any additional configuration.

Troubleshooting

Subscription alert

The Management tenant cannot be used as a data broker source tenant and this alarm is raised when trying to subscribe a Management tenant to the data broker agent.

Data broker connection error

The data broker agent is pre-configured to monitor each connector for the number of failed forwarding requests sent. If this number reaches a pre-configured threshold a CRITICAL alarm is raised in the tenant. If this happens the data will be stored until the connection is restored and it can be forwarded again. Failed requests can happen in the event the data broker subscriber tenant becomes unreachable.

Data broker slow processing alert

The data broker agent is pre-configured to monitor the rate at which events are being delivered to their destination. If events cannot be delivered fast enough, slow processing alarms will be raised. A slow processing alarm includes a connector ID to help identify which destination tenant is affected.

Queue backlog

This alarm is raised when latency for message delivery crosses a specified threshold. This usually happens if there is a backlog of undelivered events to the destination tenant due to various factors.

Average request bytes sent per second

The data broker monitors the data rate at which events are being forwarded. If this rate is lower than a pre-configured threshold, a slow processing alert will be raised. This can occur due to a slow network.

Storage quota

The storage quota is in place for a tenant when a storage quota per device is set by the platform administrator. The total storage available to the user is calculated using the formula storage quota per device x number of devices. A check is performed every night to ensure the quota is not exceeded.

In case the quota is exceeded, an email is sent to all tenant administrators to warn them that data will be deleted the following night. After 24h, if the quota is still exceeded, all data retention limits are reduced by a fixed percentage. The storage quota per device will be reduced as a result of this rule.

Example

Let us assume that a tenant has a storage quota of 10 GB. Retention rules are 80 days for measurements, 90 days for all other data.

• Day 1 - in the nightly check, the total storage is calculated at 13 GB. An email is sent to all tenant administrators.

• Day 2 - the total storage is still at 13 GB. The system determines that a 15% reduction of the retention rules is sufficient to be under the storage quota. So any measurement older than 68 days (80 days - 15%) and any other data older that 77 days (90 days - 15% results in 76.5 days, rounded to 77 days) is deleted.

The total storage is now at 9.8 GB.

Managing storage quota warning email

This feature is only visible if a storage quota was set for the tenant.

The tenant administrators can set a user group (global role) and threshold for an email to be sent once a day if the storage used is higher than a particular percentage of the storage quota. The default setup is sending an email to the “admin” role when the storage reaches 80% of maximum storage.

The email warning can also be disabled.

Enhanced time series support

The Cloud of Things Operational Store provides an enhanced time series support (so-called time series collections) for measurements data. The following section summarizes how to enable/disable this feature.

How to configure

The enhanced time series support can be configured via a REST API as a tenant configuration. The following example illustrates how to enable time series collections for a subtenant:

POST {sub-tenant-url}/tenant/options
Content-Type: application/json
{
    "category": "configuration",
    "key": "timeseries.mongodb.collections.mode",
    "value": "ENABLED"
}

The following example illustrates how to disable time series collections for a subtenant:

POST {sub-tenant-url}/tenant/options
Content-Type: application/json
{
    "category": "configuration",
    "key": "timeseries.mongodb.collections.mode",
    "value": "DISABLED"
}

Implications of the configuration

The configuration affects the collection that stores measurement data. By enabling or disabling the property, the system switches collections in the background. This might lead to a situation where data resides in multiple collections. To prevent such situations, configure the property only at the beginning of a tenant setup, ideally when no measurement data is stored yet. Migration and seamless configuration will be part of future releases.

Unsupported APIs

The following APIs are not supported and do not have a replacement:

• GET /measurement/measurements/{id}
• DEL /measurements/measurement/{id}

The following API is partially supported:

• DEL /measurements/measurement/

In release 10.16+ the parameters dateFrom and dateTo are supported and must be truncated to full hours (for example, 2022-08-19T14:00:00.000Z), otherwise an error is returned.

How to check whether time series collections are enabled

With the following request, you can check the value of the time series collections property:

GET /tenant/options/configuration/timeseries.mongodb.collections.mode
Content-Type: application/json

An example response if the configuration is enabled:

{
"category": "configuration",
"key": "timeseries.mongodb.collections.mode",
"value": "ENABLED"
}

If the configuration is not set for the tenant at all, you will get a 404 response code for the request above.