Граф коммитов

245 Коммитов

Автор SHA1 Сообщение Дата
Roberto Agostino Vitillo 29003496ba Refactor. 2015-07-28 13:50:21 +01:00
Roberto Agostino Vitillo adaba780df Add per channel-version filter options. 2015-07-26 09:41:37 +00:00
Roberto Agostino Vitillo 8b3d9877f3 Ignore negative scalar values. 2015-07-22 10:20:20 +01:00
Roberto Agostino Vitillo 794726dbee Use tornado webserver. 2015-07-21 15:48:16 +01:00
Roberto Agostino Vitillo 89f86fb9c1 Enable threading. 2015-07-21 15:30:02 +01:00
Roberto Agostino Vitillo 539db61219 Add status endpoint. 2015-07-21 13:35:24 +01:00
Roberto Agostino Vitillo f9cded82d4 Ignore histograms with negative sums. 2015-07-17 13:20:49 +01:00
Roberto Agostino Vitillo 52026a8a72 Use long literals. 2015-07-17 13:15:30 +01:00
Roberto Agostino Vitillo 78f725efc0 Close connections. 2015-07-15 07:57:40 +01:00
Roberto Agostino Vitillo 5961297a55 Merge. 2015-07-15 07:31:49 +01:00
Roberto Agostino Vitillo 199f087fd9 Enforce histogram scalar type to long. 2015-07-14 13:55:05 +01:00
Roberto Agostino Vitillo 7f7a5eddb8 Base filter options on submission_date 2015-07-09 13:29:29 +00:00
Roberto Agostino Vitillo 4479e5f7d8 Allow db name to be overridden 2015-07-08 15:44:22 +01:00
Roberto Agostino Vitillo 67319776df Use e10s runtime setting. 2015-07-07 17:30:49 +01:00
Roberto Agostino Vitillo 8a48da2653 Add support for histogram sums. 2015-07-03 11:09:40 +01:00
Roberto Agostino Vitillo 3a18e92625 Add histogram sums to aggregates. 2015-07-03 10:21:21 +01:00
Roberto Agostino Vitillo b89f1fde89 Fix double counting bug when metric is missing in the other partial aggregate. 2015-07-02 22:36:30 +00:00
Roberto Agostino Vitillo 01c136961e Undo 6bbf6d01d6 2015-07-01 13:56:28 +01:00
Roberto Agostino Vitillo 13ecea8e99 Merge branch 'master' of github.com:vitillo/python_mozaggregator 2015-07-01 12:14:22 +00:00
Roberto Agostino Vitillo 3c287d91cc Use nightly filter options on all channels. 2015-07-01 12:13:51 +00:00
Roberto Agostino Vitillo 4b030096db The ping payload can be a float apparently... 2015-07-01 09:13:05 +01:00
Roberto Agostino Vitillo 74d1d7855d Handle metric names with non printable characters 2015-07-01 08:31:21 +01:00
Roberto Agostino Vitillo 6bbf6d01d6 Fix raisy filter options retrieval. 2015-06-30 23:38:05 +01:00
Roberto Agostino Vitillo 5377ddcb85 Handle metric names with non-ascii characters... 2015-06-30 23:12:59 +01:00
Roberto Agostino Vitillo 04e519a55b Properly support flag histograms. 2015-06-26 23:21:53 +00:00
Roberto Agostino Vitillo 48a1b33312 Map json booleans to python booleans 2015-06-25 23:59:10 +00:00
Roberto Agostino Vitillo e278f3659d Add CORS 2015-06-25 23:54:31 +00:00
EC2 Default User 897be3a5a9 Add caching. 2015-06-24 21:36:11 +00:00
Roberto Agostino Vitillo 7e69f41da8 Yield 404 if filter doesn't exists. 2015-06-19 18:47:12 +01:00
Roberto Agostino Vitillo f7f097166f Combine os and osVersion filter options. 2015-06-19 18:13:38 +01:00
Roberto Agostino Vitillo 30504a90aa Use metrics stored in db. 2015-06-19 17:34:18 +01:00
Roberto Agostino Vitillo 780b45bb9d Add filter end-point. 2015-06-19 17:00:08 +01:00
Roberto Agostino Vitillo d3fc2326c7 Fix typo. 2015-06-19 12:55:21 +01:00
Roberto Agostino Vitillo 25815ccad2 Merge branch 'master' of github.com:vitillo/python_mozaggregator 2015-06-19 11:17:42 +00:00
Roberto Agostino Vitillo 9cb71672ef Add historam kind to reply. 2015-06-19 11:17:33 +00:00
Roberto Agostino Vitillo 1abc4dc4ad Clean up. 2015-06-18 15:44:54 +01:00
Roberto Agostino Vitillo 7f7e462547 Don't duplicate the histogram labels. 2015-06-18 12:24:52 +01:00
Roberto Agostino Vitillo 87abceffaa Add tests for batched queries. 2015-06-18 11:28:42 +01:00
Roberto Agostino Vitillo d3d6fcf1b2 Add support for batched queries. 2015-06-18 10:38:48 +01:00
Roberto Agostino Vitillo 6518e9ee77 Improve upsert performance. 2015-06-17 23:58:01 +01:00
Roberto Agostino Vitillo 843f9dd72d Handle more weird data. 2015-06-17 21:17:03 +01:00
Roberto Agostino Vitillo b598b926a5 Handle more missing data.
Everything that can go wrong, will go wrong!
2015-06-17 14:26:37 +01:00
Roberto Agostino Vitillo a1bdbc5a30 Use different labels for count and simple measurement histograms. 2015-06-17 11:13:06 +01:00
Roberto Agostino Vitillo 9bf6af21c1 Properly handle escape sequences. 2015-06-17 09:37:57 +01:00
Roberto Agostino Vitillo 79ff728a74 Use a raw string literal for the SQL code. 2015-06-17 08:24:44 +01:00
Roberto Agostino Vitillo 34bad3a980 Deal with more missing data... 2015-06-16 19:11:10 +01:00
Roberto Agostino Vitillo ef5cdc6635 Deal with missing histogram_type 2015-06-16 17:18:47 +01:00
Roberto Agostino Vitillo 6956cc3f5e Properly label scalar histograms. 2015-06-16 15:54:57 +01:00
Roberto Agostino Vitillo 549782fac1 Add tests for submission date aggregates. 2015-06-15 16:45:15 +01:00
Roberto Agostino Vitillo 1b06edbaf5 Add submission date support to db. 2015-06-15 15:16:17 +01:00
Roberto Agostino Vitillo f904c7b3fb Aggregate by submission date. 2015-06-15 12:00:42 +01:00
Roberto Agostino Vitillo a1bcb6c42b Clean label string. 2015-06-13 20:35:23 +00:00
Roberto Agostino Vitillo cb6cf41a53 Fix typo. 2015-06-13 19:42:37 +00:00
Roberto Agostino Vitillo b6128451e9 Fix typo. 2015-06-13 19:14:30 +00:00
Roberto Agostino Vitillo 321bdb9787 Use partial aggregation for scalar values. 2015-06-13 16:29:48 +01:00
Roberto Agostino Vitillo bc6877c28d Simplify dealing with missing values. 2015-06-13 15:57:46 +01:00
Roberto Agostino Vitillo 7c48f4a255 Add more db tests. 2015-06-13 15:32:33 +01:00
Roberto Agostino Vitillo c0f86b5a47 Merge. 2015-06-13 10:37:14 +01:00
Roberto Agostino Vitillo 9eb82a886f Kill also suprocesses after tests have completed. 2015-06-13 10:36:32 +01:00
Roberto Agostino Vitillo d1252e032c Use advisory lock to ensure atomicity. 2015-06-13 08:39:12 +01:00
Roberto Agostino Vitillo f695030892 Fix typo. 2015-06-12 22:56:25 +00:00
Roberto Agostino Vitillo e7ffe5ee07 Add bulk load. 2015-06-12 23:18:23 +01:00
Roberto Agostino Vitillo 1d16d0af97 Added more db tests. 2015-06-12 17:32:41 +01:00
Roberto Agostino Vitillo 615bd004b1 Don't aggregate over revisions. 2015-06-11 14:39:02 +01:00
Roberto Agostino Vitillo 78faba3b79 Add count histogram support. 2015-06-11 14:27:39 +01:00
Roberto Agostino Vitillo a41f3489e1 Deal with malformed histograms. 2015-06-10 18:10:55 +01:00
Roberto Agostino Vitillo 8820cfc9a7 Add e10s dimension. 2015-06-10 17:00:51 +01:00
Roberto Agostino Vitillo 5cad4a86a6 Add some tests for the Spark aggregation job. 2015-06-10 16:03:46 +01:00
Roberto Agostino Vitillo 97aa4520f1 Fix wacky escaping. 2015-06-08 19:57:22 +00:00
Roberto Agostino Vitillo 55f6fd403f Deal with missing histogram values. 2015-06-08 16:13:42 +00:00
Roberto Agostino Vitillo 415304ff3f Deal with missing histogram values. 2015-06-08 16:08:33 +00:00
Roberto Agostino Vitillo 78180e5ebc Merge branch 'master' of github.com:vitillo/python_mozaggregator 2015-06-08 15:41:46 +00:00
Roberto Agostino Vitillo c37c3b21fd Handle simple measurements histograms properly. 2015-06-08 15:41:06 +00:00
Roberto Agostino Vitillo 9d49e4617f Use proper GET parameters to specify dimensions. 2015-06-08 16:13:05 +01:00
Roberto Agostino Vitillo 7a7e5b0759 Use latest histogram definition file for each channel. 2015-06-08 14:41:26 +00:00
Roberto Agostino Vitillo 84d5da76dc Add some comments. 2015-06-08 13:50:33 +00:00
Roberto Agostino Vitillo 3aa069e192 Aggregate on multiple channels. 2015-06-08 13:32:02 +00:00
Roberto Agostino Vitillo 9bf1341f7c Update aggregator.py 2015-06-05 18:02:05 +01:00
Roberto Agostino Vitillo 592ddcfdf4 Ignore histograms with empty values dict. 2015-06-05 14:32:00 +00:00
Roberto Agostino Vitillo 830c98aada Fix incomplete histograms for simple measurements. 2015-06-05 13:37:25 +00:00
Roberto Agostino Vitillo b052d743f5 Transact upserts for aggregates within the same channel, version and buildid to ease recovery in case of failure. 2015-06-04 14:27:09 +00:00
Roberto Agostino Vitillo 3eca771167 Add submission date to the mapper's key 2015-06-04 11:11:36 +00:00
Roberto Agostino Vitillo 8f5e0f2eaa Fix escape character bugs. 2015-06-04 09:38:27 +00:00
Roberto Agostino Vitillo c296e15fa6 Refactor query execution and use a new connection for each request 2015-06-03 15:24:37 +00:00
Roberto Agostino Vitillo 1e24f4f3ff Add arguments to service. 2015-06-03 13:02:49 +00:00
Roberto Agostino Vitillo f51ca2ef1e Add db service. 2015-06-01 17:51:24 +01:00
Roberto Agostino Vitillo ac476ac764 Add stored procedure to list all channels. 2015-06-01 13:31:15 +00:00
Roberto Agostino Vitillo cc0d77eb90 Add query to list all buildids for a given channel. 2015-06-01 13:10:41 +00:00
Roberto Agostino Vitillo 7574824eae Use RDS API to read db parameters. 2015-06-01 12:33:55 +00:00
Roberto Agostino Vitillo 1a3d8ae30d Add query API. 2015-05-29 14:21:15 +00:00
Roberto Agostino Vitillo 94b50ac641 Improve aggregation performance. 2015-05-28 23:02:45 +00:00
Roberto Agostino Vitillo ef7dbe99bf Remove whitespace. 2015-05-28 20:21:17 +00:00
Roberto Agostino Vitillo f311578ad8 Use new aggregation algorithm. 2015-05-28 13:05:11 +00:00
Roberto Agostino Vitillo 92cac3b91e Deal with new aggregation format. 2015-05-27 23:02:07 +00:00
Roberto Agostino Vitillo ccb7453d9e Add experimental aggregator. 2015-05-27 21:54:35 +00:00