bigquery-etl/sql/mozfun/event_analysis
Rhys 1ace0fe2b7
Ran YAMLlint on all yaml files and resolved linting issues (fixes #1297) (#1481)
* "Ran YAMLlint on all yaml files"

* "Moved product info metadata table to README file"

* "Reformatted yaml lists"

* "Updated line breaks so script runs"

* "Updated line breaks so script runs"

* "Undid line breaks"

* "Created custom config file"

* "Removed base document id"

* "Undid line breaks"

* "Reformatted code"

* "Trimmed whitespace"

* "Undid line break"

* "Introduced newline"

* "Trimmed whitespace"

* "Added yamillint to config file"

* "Added yamllint to config file"

* "Moved up yamllint test"

* "Trimmed whitespace"

* "Trimmed whitespace"

* "Trimmed whitespace"

* "Trimmed whitespace"

* "Removing hyphen to fix CI error"

* "Indentation to remove CI error"

* "Included yamllint install in build run"

* "Added yamllint in requirements.txt and .in file"

* "Moved install yamllint step to its own stage"

* "Updated yamllint test"

* "Updated circleci step"

* "Reformatted code"

* "Added yamllint to circleci steps"

* "Added checkout block to yamllint step"

* "Trimmed whitespace"

* "Undid yamllint step"

* "Specified directory name for yamllint test"

* "Fixed yamlint errors"

* "Fixed yamllint errors"

* "Fixed yamllint errors"

* "Fixed yamllint errors"

* "Ignore pathway in linting"

* "Added ignore venv pathway during linting"

* "Updated ignore block"

* "Updated ignore block"

* "Removed ignore block"

* "Updated ignore block"

* "Indented base as a list"

* "Indented base item"

* Update tests/sql/moz-fx-data-shared-prod/search_derived/mobile_search_clients_last_seen_v1/test_day_bit_shifting/expect.yaml

Co-authored-by: Anthony Miyaguchi <acmiyaguchi@gmail.com>

* "Resolved linting errors"

* "Referenced tables put back on same line"

* "Fixed linting error"

* Update sql/moz-fx-data-shared-prod/account_ecosystem_derived/fxa_logging_users_daily_v1/metadata.yaml

Co-authored-by: Anthony Miyaguchi <acmiyaguchi@gmail.com>

* "Fixed linting error"

Co-authored-by: Anthony Miyaguchi <acmiyaguchi@gmail.com>
2020-10-29 17:24:55 -07:00
..
aggregate_match_strings Ran YAMLlint on all yaml files and resolved linting issues (fixes #1297) (#1481) 2020-10-29 17:24:55 -07:00
create_count_steps_query Ran YAMLlint on all yaml files and resolved linting issues (fixes #1297) (#1481) 2020-10-29 17:24:55 -07:00
create_events_view Ran YAMLlint on all yaml files and resolved linting issues (fixes #1297) (#1481) 2020-10-29 17:24:55 -07:00
create_funnel_regex Ran YAMLlint on all yaml files and resolved linting issues (fixes #1297) (#1481) 2020-10-29 17:24:55 -07:00
create_funnel_steps_query Ran YAMLlint on all yaml files and resolved linting issues (fixes #1297) (#1481) 2020-10-29 17:24:55 -07:00
escape_metachars Ran YAMLlint on all yaml files and resolved linting issues (fixes #1297) (#1481) 2020-10-29 17:24:55 -07:00
event_index_to_match_string Ran YAMLlint on all yaml files and resolved linting issues (fixes #1297) (#1481) 2020-10-29 17:24:55 -07:00
extract_event_counts Ran YAMLlint on all yaml files and resolved linting issues (fixes #1297) (#1481) 2020-10-29 17:24:55 -07:00
extract_event_counts_with_properties Ran YAMLlint on all yaml files and resolved linting issues (fixes #1297) (#1481) 2020-10-29 17:24:55 -07:00
get_count_sql Ran YAMLlint on all yaml files and resolved linting issues (fixes #1297) (#1481) 2020-10-29 17:24:55 -07:00
get_funnel_steps_sql Ran YAMLlint on all yaml files and resolved linting issues (fixes #1297) (#1481) 2020-10-29 17:24:55 -07:00
README.md Change event_analysis header 2020-10-06 10:54:25 -04:00

README.md

event_analysis

These functions are specific for use with the events_daily and event_types tables. By themselves, these two tables are nearly impossible to use since the event history is compressed; however, these stored procedures should make the data accessible.

The events_daily table is created as a result of two steps:

  1. Map each event to a single UTF8 char which will represent it
  2. Group each client-day and store a string that records, using the compressed format, that clients' event history for that day. The characters are ordered by the timestamp which they appeared that day.

The best way to access this data is to create a view to do the heavy lifting. For example, to see which clients completed a certain action, you can create a view using these functions that knows what that action's representation is (using the compressed mapping from 1.) and create a regex string that checks for the presence of that event. The view makes this transparent, and allows users to simply query a boolean field representing the presence of that event on that day.