Граф коммитов

84 Коммитов

Автор SHA1 Сообщение Дата
Sridhar Vemula 9c6030bd1c
using goreleaser for semver builds on sloop. (#121)
* using goreleaser for semver builds on sloop. Integrated with travis ci and docker.

* fix: deploy script update to ignore new lines

* fix: deploy script update to ignore new lines

* fix: using make target instead of shell script as travis having issues with new lines

* fix: removed travis environment variable from makefile

* fix: updated build script
2020-07-08 12:00:58 -07:00
Philippe scorsolini b6b19fc13a
yaml list elements fix (#123)
Just a visual fix, but I like to decrease the entropy in the universe 🙏
2020-06-29 08:29:03 -07:00
Sridhar Vemula 8f4c702326
fixed scrolling issue on the sideleftnav (#122) 2020-06-24 13:27:23 -07:00
sana-jawad 6874a7cc5f
Read me for memory consumption (#119)
* Added guidance for memory consuption

* Added metric for dropPrefix latency so that we can measure for how much time it locks the DB

* fixed the comment
2020-05-11 16:13:05 -07:00
cmeister-sfdc 245ccdd3c4
Use CRD's specified kind for event listening on CRDs (#115) 2020-05-11 14:33:24 -07:00
sana-jawad 968799a319
Updated number of version to keep to 1 (#117) 2020-05-06 13:50:06 -07:00
sana-jawad c595202d01
Sloop was not able to reduce the size on disk after GC runs. This also plays a big role in memory consumption. (#114)
Upgraded Badger version.
Added flattening at start up time.
Fixed the event count spreading issue which resulted in uneven data distribution across partitions.
Moved to drop prefix as it yields better space claim.
Added feature flag for switching to delete prefix. Also changed the numberfversions to 0 so that delete prefix would reclaim space. dgraph-io/badger#1228
Fixed the issue of unclaimed !badger!move prefixes which are never cleaned up. Details: dgraph-io/badger#1288
Added support in debugging pages to see internal keys.
2020-04-24 11:32:51 -07:00
sana-jawad 3506d40246
Garbage collection fix (#110)
* Garbage collection fix

Issues fixed:

1. Garbage collection was not able to sustain the required size limit.
2. Exposed more badger options to help in compression
3. Fixed runtime bug encountered when event records with nil map was retrieved.

* Incorporated comments

* Incorporated Duke's comments
2020-03-13 15:16:26 -07:00
sana-jawad c29fcb8723
Sloop keys histogram (#109)
Added Sloop keys Histogram for debug support
2020-02-28 12:23:37 -08:00
sana-jawad c7ebc8596f
Added readme for tables in Sloop (#108)
Added readme for tables in Sloop
2020-02-20 12:42:41 -08:00
sana-jawad 7d94723dfa
Exposed more badger options for memory utilization tweaks (#107)
* GC Fix

* Addressed all comments and added unit test

Fixed the replay use case and added unit test

* Removed the GetMin and GetMaxPartitions functions as they are not required now

* Added more unit test cases

* Badger Tweaks for memory utilization
2020-02-19 14:57:22 -08:00
sana-jawad e0a8d06841
Drop prefix optimization: Replaced DropPrefix with deletion of keys. Also added some more metrics. (#105)
* GC Fix

* Addressed all comments and added unit test

Fixed the replay use case and added unit test

* Removed the GetMin and GetMaxPartitions functions as they are not required now

* Added more unit test cases

* Removed dropPrefix to fix the performance

* Added some more metrics

* Added unit tests and some more metrics

* Removed dropPrefix to fix the performance

* Added some more metrics

* Added unit tests and some more metrics

* Incorporated some comments

* Incorporated all the comments

* fixed the minor comment
2020-02-12 17:07:00 -08:00
Thomas Hargrove d6f04a9f7c
Update README.md (#104) 2020-02-05 22:59:38 +05:30
sana-jawad 326c1194e3
Fix for Sloop garbage collection when its not able to successfully decrease the size on disk (#100)
Solved Problem: Disk usage in our cluster grew way beyond the config limit resulting the application to be OOM killed. There were a couple of issues identified and fixed as follows:

1. Event count table data (although it is the smallest table in terms of number of keys and size) was getting added beyond the max look back time.

2. The sprinkling of event count data was using max look back as a boundary resulting in small partitions containing very less event count data spread all across till look back time.

3. Garbage collection was never called for watch activity table.

4. Added prometheus Metrics for better visibility.
2020-02-04 10:23:33 -08:00
Thomas Hargrove ba2c651886
Add better badger debugging (#101) 2020-01-31 00:08:34 +05:30
mtmn a9469f43b5 [providers/aws/sloop-to-eks.sh] Add 'profile' parameter (#95) 2020-01-24 05:00:46 +05:30
Thomas Hargrove 4b0a27ee78
Tweak Badger GC and expose several Badger config options (#94)
* Add value log GC

* Add various badger options and tweak our GC

* Add more metrics and PR comments

* PR Comments

* Fix metrics

* Fix latency metrics

* Minot comment tweaks
2020-01-07 23:59:59 +05:30
DuncanSmith1126 6ec9eb4f30 Update badger to v2 (#92) 2020-01-04 00:38:09 +05:30
Ash Caire 93fce49aed Use Values.name for configMap name (#91)
The created config map uses Values.name, so the pod reference
should also use it otherwise the pod fails to start
2019-12-11 09:51:09 -08:00
DuncanSmith1126 9a32526aa8 Updating screenshot (#89) 2019-12-09 16:03:09 -08:00
DuncanSmith1126 7c160d5576
Change to dark mode (#87)
* Change to dark mode

* Fixes

* more fixes
2019-12-09 14:46:54 -08:00
hsiddulugari f9832284e1 Added aws-iam-authenticator to docker and a launch script (#73)
* Add aws-iam-authenticator to docker and a launch script

* Move sloop-to-eks.sh to provider specific dir
2019-11-21 18:34:27 -08:00
Min Wang 5fc97a7211 Update README for slack info (#84)
* Update README.md

* Update README.md

* Update README.md
2019-11-21 16:14:02 -08:00
Ryan Brainard add645bce3 Database Restore (#76)
* Add db.Load() to interface, impl, mock

* Add Database Restore Feature

* Document backup and restore

* Remove DB restore test
2019-11-18 16:12:26 -08:00
Ryan Brainard d3cfa2900c Only require valid context when watching (#75)
* Split MakeKubernetesClient into Client & Context as to avoid -disable-kube-watch from requiring a valid context

Fixes https://github.com/salesforce/sloop/issues/58

* Put back kubeContext into log line
2019-11-18 15:37:22 -08:00
Thomas Hargrove 760cad8087
Fix diff in inner html (#80) 2019-11-18 12:07:31 -08:00
Duke Harlan 8e48c18490 attempting to repair damage from previous padding change (#79)
* attempting to repair damage from previous padding change

* address PR comment
2019-11-18 11:43:16 -08:00
Thomas Hargrove d03755ab0b Clean up logo and make it smaller (#78) 2019-11-18 10:13:48 -08:00
Thomas Hargrove 051a1283e5
Add logo to readme (#77) 2019-11-18 09:32:11 -08:00
Ryan Brainard 6788033426 Update readme to link to Docker image (#74)
* Update readme to link to Docker image

* Remove comment from Helm chart about user setting their own image
2019-11-18 09:10:10 -08:00
Min Wang 79d4efcad4
Add the payload diff feature (#72)
* Add the payload diff feature

* address comments
2019-11-17 13:05:23 -08:00
Min Wang 5c23139bdc
Add payload json blob (#68)
* display payload json

* fix rebase conflict

* fixed nits

* remove unused files

* address comments
2019-11-15 13:37:49 -08:00
Thomas Hargrove 847119c2c4
Fix race condition in unit test (#71) 2019-11-15 13:31:35 -08:00
Duke Harlan 9e42aca84a
move padding in to resource.html - duplicate behavior in multiple uses (#70) 2019-11-15 13:28:08 -08:00
nurland 5b5f9064ae Enable auto push of docker images (#69)
* Enable auto push of docker images

* Fix Makefile
2019-11-15 13:17:31 -08:00
Thomas Hargrove 66bdd92f39 Fix defaultLookback if not set in config (#67) 2019-11-14 17:15:45 -08:00
Thomas Hargrove d4144e95ea
Remove dupe events for details page (#66)
* Remove dupe events for details page

* Fix non-namespaces resources

* PR feedback

* PR Feedback
2019-11-14 16:18:37 -08:00
nurland b9dd316ece Update helm instructions and add Docker Pulls badge (#65) 2019-11-14 15:03:50 -08:00
Thomas Hargrove 4cb746ddd9
Add precise time filtering to event and payload views (#64)
* Properly compute time ranges for resource details page

* Fix typos

* Add unit tests

* PR Feedback
2019-11-14 15:01:10 -08:00
nurland 207e913bd8
Update helm chart to use public Sloop image (#63) 2019-11-14 13:03:10 -08:00
DuncanSmith1126 18de6643a5 Fixed author and resource detail page (#59) 2019-11-14 12:49:21 -08:00
Ryan Brainard 630fbc9d83 Fix Helm service name and port forwarding (#60)
* Fix Helm service selector to use label name

* Fix port forward instructions to use service
2019-11-14 10:12:15 -08:00
Thomas Hargrove fec78f6851 Make architecture image smaller (#61) 2019-11-14 10:02:53 -08:00
Min Wang 2586192ac7
fix the misplacement of code caused by go gen (#55) 2019-11-12 17:25:09 -08:00
Ryan Brainard 6bf8712bad Data Backup Endpoint (#56)
* Add /data/backup endpoint to dump database

* Use tables.Db() instead of raw db

* Add backupHandler doc

* Provide better error message for since parsing issues
2019-11-12 17:15:26 -08:00
Duke Harlan c54c004c25 add top level sort of 'kind' (#57) 2019-11-12 17:07:24 -08:00
Min Wang bc8bbe15e6
Add Payload frontend (#54)
* wip

* Show payloads during window time

This PR adds getPreviousKey logic in getPayload logic.
Add front end to show the payloads during the windown time.

* address comments

* fix the unused code
2019-11-08 17:19:46 -08:00
Thomas Hargrove ae699affed
Add new logo (#53)
* Add new logo

* Fix file extension
2019-11-04 10:34:57 -08:00
Thomas Hargrove da0ba06f40
Update archicture diagram (#48) 2019-10-22 10:14:57 -07:00
Thomas Hargrove ae6842736b
Add go report card and fix some errors (#49) 2019-10-22 10:14:47 -07:00