Stream Analytics

Zoho Analytics supports real-time data ingestion from streaming services, enabling continuous monitoring and analysis. Currently, it supports the following methods:

1. REST API
2. Google Pub/Sub Push Subscription

This functionality allows users to ingest live data, track critical events, and generate real-time insights. It is particularly useful for monitoring financial transactions, user activity, sensor data, and other time-sensitive information. Whether using custom APIs or Google Pub/Sub push subscriptions, Zoho Analytics seamlessly integrates streaming data with existing datasets, facilitating timely, data-driven decisions.

Report Refresh Rate for Stream Data

Reports created using streaming data refresh automatically every 30 seconds, ensuring up-to-date insights. For more details, click here.

Note: Zoho Analytics can store up to 1 million rows or 1 hour of streamed data at any given time.

On this page

• Streaming data integration
  • API Integration
  • Google Pub/Sub Push Subscription
• Finalizing the Streaming Configuration
• Managing Data Sources
• Frequently Asked Questions

Streaming Data Integration

1. API Integration

Zoho Analytics enables seamless streaming data integration via REST APIs, simplifying data management and real-time analytics.

Steps to Import Streaming Data via API

1. Navigate to the Import Your Data page and select API under the Streaming Data section.
   
2. In the API Streaming Data dialog, enter the Workspace Name and an optional Workspace Description.
3. Choose how to design the streaming table:
   1. Provide Sample Data: Paste a sample JSON, and Zoho Analytics will automatically create the table.
   2. Design the Table: Manually define column names and properties.
      

Using the "Provide Sample Data" Option

1. Paste the sample JSON in the Provide Sample JSON field and click Next.
   
2. Review and modify the generated table structure.
3. Enter the Table Name and optional Table Description. Set the Date Format for the streamed data; this format will be applied to all the date columns in the table.
   
4. The table metadata is created automatically based on the provided sample JSON.
   

Using the "Design the Table" Option

1. Select the Design the Table option and click Next.
   
2. Enter the Table Name, optional Table Description, and set the Date Format.
3. Specify columns and their data types to establish the table’s metadata.
   

Additional Column Properties:

• Is Mandatory: Ensures the column is required during import.
• Default Value: Assigns a default value if the field is empty.
• Is Personal Data: Marks the column as personal data, ensuring encryption in Zoho Analytics.
  

Final Steps

• Click Next to create the streaming model.
• Copy the API URL and use it in your service endpoint to push data into Zoho Analytics.
  

2. Google Pub/Sub Push Subscription Integration

Zoho Analytics allows integration with Google Pub/Sub Push Subscription for real-time data streaming. Learn more about Google Pub/Sub Push Subscription.

Steps to Import Streaming Data via Google Pub/Sub

1. Go to the Import Your Data page. Under Streaming Data, select Google Pub/Sub Push Subscription.
   
2. In the Google Pub/Sub Streaming Data dialog, enter the Workspace Name and an optional Workspace Description.
3. Choose how to design the streaming table:
   1. Provide Sample Data: Paste a sample JSON, and Zoho Analytics will automatically generate the table structure.
   2. Design the Table: Manually define columns and properties.
      

Using the "Provide Sample Data" Option

1. Paste the sample JSON in the Provide Sample JSON field and click Next.
   
2. Enter the Table Name, optional Table Description, and set the Date Format for the streamed data.
   
3. Review and edit the automatically generated table.
4. The table metadata is created based on the provided sample JSON.
   

Using the "Design the Table" Option

1. Select the Design the Table option and click Next.
   
2. Enter the Table Name, optional Table Description, and set the Date Format.
3. Specify columns and their data types to establish the table’s metadata.
   

Additional Column Properties:

• Is Mandatory: Marks a column as required during import.
• Default Value: Specifies a default value if the field is empty.
• Is Personal Data: Marks the column as personal data, ensuring encryption in Zoho Analytics.
  

Final Steps

• Click Next to create the streaming model.
• Copy the Push URL and configure the end point in Google Pub/Sub to push data into Zoho Analytics.
  

Finalizing the Streaming Configuration

Once the streaming model is created, a unique URL is generated. This URL points to the streaming table in Zoho Analytics. The user must configure this URL at their required streaming service endpoint to enable seamless data ingestion into Zoho Analytics.

Steps to Configure the Streaming URL

1. Configure the URL at the Streaming Service
   • The generated URL should be set up at the data source (API service or Google Pub/Sub Push Subscription) to ensure data is being pushed into Zoho Analytics.
     
2. Validate the Streaming Connection
   • After configuring the URL, users must validate whether the streaming service is correctly pushing data into Zoho Analytics.
   • Zoho Analytics will monitor the URL for 120 seconds to check if data is received in the defined table format, and the status will be shown as Listening.
     
   • If no data is received within this period, users can retry the validation process.
3. Monitor the Streaming Status
   • If data is received successfully, the status will change to "Consuming", indicating that the streaming is active and operational.
     
   • If there are any issues or failures, the status will be updated to "Failed", along with a specific error message detailing the problem.
     
4. Validating the URL in Data Sources
   • Users can also validate the streaming URL directly from the Data Sources page in Zoho Analytics.
   • The validation behavior remains the same—Zoho Analytics will listen for incoming data for 120 seconds, and the status will update accordingly.

These steps ensure that streaming data is properly configured, validated, and monitored for seamless integration into Zoho Analytics.

Managing Data Sources

To manage streaming data sources:

1. Click the Data Sources tab in the left navigation pane.
2. The Data Sources page will open, displaying the following details:
   • Table Name: Name of the table receiving live data.
   • Push API URL: The URL currently in use for data streaming.
   • Last Data Received Time: Timestamp of the most recent data received.
   • Data Streaming Status: Indicates if the streaming is active. If functioning correctly, the status will be "Consuming". To learn about the other data streaming status, click here.
     
3. Regenerate Key - Zoho Analytics provides a Regenerate Key option, which generates a new secret key for the endpoint. Users can utilize this option in scenarios such as:
   • Changing the Data Source: If the source is updated, users can regenerate the key and update the endpoint accordingly.
   • Security Reasons: Users may regenerate the key to enhance security and prevent unauthorized access.
   • Once the key is regenerated, the updated secret key must be reconfigured at the streaming service endpoint to ensure uninterrupted data ingestion.

Frequently Asked Questions

1. Will reports in the dashboard be refreshed automatically with streaming data?

No, reports in dashboards will not be automatically refreshed when streaming data is updated. While reports created using streaming data are refreshed every 30 seconds, dashboards must be manually refreshed to reflect the latest data.

2. What is the time period for validation, and what happens if validation fails?

Zoho Analytics will listen for 120 seconds after the user clicks the Validate button to check if data is received in the defined table format.

• If the data is received within this time, the status will change to "Consuming".
• If no data is received within this period, the validation will fail, and users will have to retry the validation process.
• If the validation repeatedly fails, users should check the endpoint configuration and ensure that the data is being sent in the correct format.

3. What are the different streaming statuses in Zoho Analytics?

Zoho Analytics provides multiple statuses to indicate the state of the streaming process:

• Yet to Validate – The URL is not yet configured with any data source.
• Listening – Zoho Analytics is currently validating the connection after the user clicks the Validate button.
• Consuming – Data is successfully being streamed in continuously.
• Paused – Streaming is manually paused by the user. A Resume button will be available to restart the stream.
• Failed – The connection has failed, or the last 10 messages processed encountered errors due to a mismatch with the data model. The failure status will include error codes and reasons.

4. How much streaming data is stored in Zoho Analytics?

Zoho Analytics retains up to 1 million rows or 1 hour of streamed data at any given time. Data beyond this limit is automatically discarded.

5. Can I regenerate the API key for security or source updates?

Yes, Zoho Analytics provides a Regenerate Key option, which generates a new secret key for the endpoint. This is useful when:

• The data source changes, and the new source needs to be configured.
• For security reasons, to prevent unauthorized access.

Once the key is regenerated, users must update it at the streaming service endpoint to continue streaming data.