Knowi enables data discovery, visualization, data manipulation, warehousing and reporting automation from Google Analytics, along with the ability to merge that data with other data stores.
If you are not a current Knowi user, check out our Google Analytics Instant Reporting page to get started.
Connecting
The following GIF image shows how to connect to Google Analytics.
-
Login to Knowi and select Settings -> Datasources from the left down menu.
-
Click on Google Analytics. An authentication popup will be shown. Sign in to your Google Analytics account to complete the OAuth process, where a secure token is exchanged between Google Analytics and Knowi.
-
After the authorization, select a Google Analytics Profile ID set up the datasource against.
-
Save the Connection. Click on the "Configure Queries" link on the success bar.
Alternatively, Use our Cloud9Agent to connect to your Google Analytics account. This can securely pull data inside your network. See agent configuration for more details.
Queries & Reports
-
This section enables you to set up automated data pulls from Google Analytics that can either be run on a schedule or once.
Report Name: Specify a name for the report.
Metrics : Select a list of metrics you want to track from the drop down (or type in)
Dimensions : Dimensions enable grouping of data for the metrics selected. For example, ga:date groups the metrics selected on a daily basis.
Max Results: Maximum number of records to pull. Note: Google Analytics allows up to 10k rows per data pull.
Sort By Field: Field sorting for the data returned. Example: Ascending: ga:visits Descending: -ga:visits
Cloud9QL Query: Optional syntax that can be applied to the results of the Google Analytics data to further transform the results. Example:
select sum(ga:visits) as Visits, sum(ga:newVisits) as New visits,month(ga:date) as Month group by Month(ga:date) order by Month asc
This groups Visits and New Visits on a monthly basis (with custom labels). Detailed Cloud9QL documentation
Date Range: Specify a date range to pull data from. Use a number followed by the date identifier. Examples:
Date Range Op Comments min Date range of n minutes back from now. Example (10 minutes) : 10min d Date range of n days back from today. Example (up to 120 days) : 120d w Date range of n weeks back from today. Example (up to 10 weeks): 10w m Date range of n months back from today. Example (up to 3 months): 3m y Date range of n years back from today. Example (up to last 1 year): 1y today Midnight of today till now yesterday Midnight of yesterday till now this hour Current hour till now this week Midnight of Monday of the current week till now this month Midnight of the 1st of the month, to now till now last hour Last hour, adjusted to 0 mins and 0 secs till now last week Last Week Monday, adjusted to Midnight till now last month Last Month, adjusted to the first of that month, to now.
-
Scheduling: Configure how often this should be run. Select 'None' for a one-time operation. The results are stored in your warehouse in Knowi.
-
Overwrite Strategy (for scheduled query runs):
This determines how the data is stored in your warehouse in Knowi:
i. If empty, data will be added on to the existing data for this dataset within Knowi. Or,
ii. "All": Any existing data for this dataset will be replaced by this results.
iii. One or More Field Names (Example: "A,B,C"): A new record is created where the values of the combination of the field names do not exist, and, updates current records for the field grouping where it exists. For example, if this is set to say "Date, Type", existing data with the same Date and Type values will be updated with the latest data, and new records created when they do not exist.
-
Click 'Preview' to see the results.
-
Click on 'Save' to complete setting up the report. This also sets up this data extraction on a schedule, if configured.
-
Click on 'Dashboards' to access dashboards. You can drag and drop the newly created report from the bottom list into to the dashboard.
Cloud9Agent Configuration
Use Cloud9Agent as an alternative to UI based connectivity outlined above. The agent runs inside your network to extract data from Google Analytics and sends the extracted/manipulated data into your Knowi warehouse. Check out Cloud9Agent to download your agent.
For sample Google Analytics configuration, see the datasource_example_googleanalytics.json and query_example_googleanalytics.json examples folder under the Cloud9Agent install directory.
Highlights:
- Connects to Google Analytics using OAuth tokens
-
Pulls data using GA API, with optional manipulations using Cloud9QL
Obtain the Refresh token using the Connect step in the UI section above.
Datasource Configuration:
Parameter | Comments |
---|---|
name | Unique Datasource Name. |
datasource | Set value to ga |
authRefreshToken | OAuth Offline Token generated by Google Analytics |
gaProfileID | Profile ID is a Google Unique identifier for the account. To determine profile ID, login to your Google Analytics account. The URL when you login will have a structure similar to: https://www.google.com/analytics/web/#report/visitors-overview/a5559982w55599512p12345678. In the structure at the end of the URL, the 8 digits that follow 'p' is your profile id (12345678 in the example URL) |
Query Configuration:
Query Config Params | Comments |
---|---|
entityName | Dataset Name Identifier |
identifier | A unique identifier for the dataset. Either identifier or entityName must be specified. |
dsName | Name of the datasource name configured in the datasource_XXX.json file to execute the query against. Required. |
gaStartDate | Begin Date to pull data for, in yyyy-MM-dd format. Either gaStartDate and gaEndDate, OR, gaDateRange must be specified |
gaEndDate | End Date to pull data for, in yyyy-MM-dd format. Either gaStartDate and gaEndDate, OR, gaDateRange must be specified |
gaDateRange | Date Range. Takes precedence over gaStartDate or gaEndDate, if specified. Example: 5d, this week, etc. See all supported dateRange params below |
gaMetrics | Required. Metrics to track. Multiple metrics can be tracked using a comma delimiter. Example: ga:visitors,ga:newVisits. See GA API Explorer, metrics section for supported metrics. |
gaDimensions | Dimensions/Grouping on the data. Multiple dimensions can be specified by comma delimiter. See GA API Explorer, dimensions section for supported dimensions. |
gaMaxResults | Required. Maximum number of records to pull |
gaSort | Optional. Sorting order: +/-, minus indicates descending, + ascending. Example: -ga:date |
c9QLFilter | Optional. Can be used to manipulate to results retrieved from GA API. See Cloud9QL docs |
frequencyType | Scheduling frequency type. One of minutes, hours, days, weeks, months. If this is not specified, this is treated as a one-time query, executed upon Cloud9Agent startup (or when the query is first saved) |
frequency | Indicates the frequency if frequencyType is defined. For example, if this value is 10 and the frequencyType is minutes, the query will be executed every 10 minutes |
startTime | (Optional) Can be used to specify when the query should be run for the first time. If set, the frequency will be determined from that time onwards. For example, is a weekly run is scheduled to start at 07/01/2014 13:30, the first run will run on 07/01 at 13:30, with the next run at the same time on 07/08/2014. The time is based on the local time of the machine running the Agent. Supported Date Formats: MM/dd/yyyy HH:mm, MM/dd/yy HH:mm, MM/dd/yyyy, MM/dd/yy, HH:mm:ss,HH:mm,mm |
overrideVals | This enables data storage strategies to be specified. If this is not defined, the results of the query are added to the existing dataset. To replace all data for this dataset within Knowi, specify {"replaceAll":true}. To upsert data specify "replaceValuesForKey":["fieldA","fieldB"]. This will replace all existing records in Knowi with the same fieldA and fieldB with the current data and insert records where they are not present. |
Date Range Op | Comments |
---|---|
min | Date range of n minutes back from now. Example (10 minutes) : 10min |
d | Date range of n days back from today. Example (up to 120 days) : 120d |
w | Date range of n weeks back from today. Example (up to 10 weeks): 10w |
m | Date range of n months back from today. Example (up to 3 months): 3m |
y | Date range of n years back from today. Example (up to last 1 year): 1y |
today | Midnight of today till now |
yesterday | Midnight of yesterday till now |
this hour | Current hour till now |
this week | Midnight of Monday of the current week till now |
this month | Midnight of the 1st of the month, to now till now |
last hour | Last hour, adjusted to 0 mins and 0 secs till now |
last week | Last Week Monday, adjusted to Midnight till now |
last month | Last Month, adjusted to the first of that month, to now. |
Datasource Example:
[
{
"name":"demoGA",
"datasource":"ga",
"authToken":"1/JL-_Wkuan6c_Hs5FdEkgGMJ4wCgHm4STz2J7EgjN4OI",
"gaProfileID":"ga:66465881"
}
]
Query Example:
[
{
"entityName":"Website Visits",
"dsName":"demoGA",
"gaMetrics":"ga:visitors,ga:newVisits",
"gaDimensions":"ga:date",
"gaDateRange":"10d",
"gaMaxResults":"1000",
"gaSort":"-ga:date",
"c9QLFilter":"select ga:visitors as Visitors, ga:newVisits as New Visits, ga:date as Date",
"overrideVals":{
"replaceAll":true
},
"frequencyType":"hour",
"frequency":2
}
]