* Implement input plugin setup + perf and containerinventory plugins (#1112)
* migrate perf and containterinventory plugins
---------
Co-authored-by: Amol Agrawal <amagraw@microsoft.com>
* add var to livenessprobe (#1116)
Co-authored-by: Amol Agrawal <amagraw@microsoft.com>
* Add perf and container inventory plugins for Windows and integrating them to named pipe (#1113)
Add perf and container inventory plugins for Windows and integrating them to named pipe (#1113)
* Resolving comments
* Solving a bug
* Using the new input plugins only for windows msi mode
* Remove the duplicate flow of logs
* Not update the config files when legacy and resource optimization is on
* Not updating the config map for geneva as well
* Solving timestamp bug and add windows_rs.conf files
* Var name change bug
* Use the stat of a file instead of reading from file
* address PR comments - I
* update fluent-bit to 2.1.10
* update trivyignore
* fix bug in confg parsing
* add shared library
* revert fluent-bit upgrade in windows
* Adding a configmap option for disabling windows fluent-bit
* Adding a configmap option for disabling windows fluent-bit
* Solving a bug
* Resolving comments
* Solving RS bug
* Solving a bug
* Resolving comments
* remove linux option from cm
* Update trivyignore
* Sign the containerinventory.so and perf.so files
* Signing windows binary files
* Update the go mod tidy for the ci_prod changes
* Update go.mod to 1.21
* bug
* Update fluent-bit conf
* Update the fluent-bit conf file
* fluentd starting two times
---------
Co-authored-by: Amol Agrawal <pfrcks@gmail.com>
Co-authored-by: Amol Agrawal <amagraw@microsoft.com>
Co-authored-by: Janvi Jatakia (from Dev Box) <jajataki@microsoft.com>
* adding Application Insights webhook image to the CI pipeline - build and release
---------
Co-authored-by: ramanujbhattacharjee93 <rambhatt@microsoft.com>
* rename omsagent to ama-logs
* undo merge change related errors
* workaround for build
* add change to windows container name
* workaround fixed?
Co-authored-by: Amol Agrawal <amagraw@microsoft.com>
* separate build yamls for ci_prod branch (#415)
* re-enable adx path (#420)
* Gangams/release changes (#419)
* updates related to release
* updates related to release
* fix the incorrect version
* fix pr feedback
* fix some typos in the release notes
* fix for zero filled metrics (#423)
* consolidate windows agent image docker files (#422)
* consolidate windows agent image docker files
* revert docker file consolidation
* revert readme updates
* merge back windows dockerfiles
* image tag update
* Gangams/cluster creation scripts (#414)
* onprem k8s script
* script updates
* scripts for creating non-aks clusters
* fix minor text update
* updates
* script updates
* fix
* script updates
* fix scripts to install docker
* fix: Pin to a particular version of ltsc2019 by SHA (#427)
* enable collecting npm metrics (optionally) (#425)
* enable collecting npm metrics (optionally)
* fix default enrichment value
* fix adx
* Saaror patch 3 (#426)
* Create README.MD
Creating content for Kubecon lab
* Update README.MD
* Update README.MD
* Gangams/add containerd support to windows agent (#428)
* wip
* wip
* wip
* wip
* bug fix related to uri
* wip
* wip
* fix bug with ignore cert validation
* logic to ignore cert validation
* minor
* fix minor debug log issue
* improve log message
* debug message
* fix bug with nullorempty check
* remove debug statements
* refactor parsers
* add debug message
* clean up
* chart updates
* fix formatting issues
* Gangams/arc k8s metrics (#413)
* cluster identity token
* wip
* fix exception
* fix exceptions
* fix exception
* fix bug
* fix bug
* minor update
* refactor the code
* more refactoring
* fix bug
* typo fix
* fix typo
* wait for 1min after token renewal request
* add proxy support for arc k8s mdm endpoint
* avoid additional get call
* minor line ending fix
* wip
* have separate log for arc k8s cluster identity
* fix bug on creating crd resource
* remove update permission since not required
* fixed some bugs
* fix pr feedback
* remove list since its not required
* fix: Reverting back to ltsc2019 tag (#429)
* more kubelet metrics (#430)
* more kubelet metrics
* celan up new config
* fix nom issue when config is empty (#432)
* support multiple docker paths when docker root is updated thru knode (#433)
* Gangams/doc and other related updates (#434)
* bring back nodeslector changes for windows agent ds
* readme updates
* chart updates for azure cluster resourceid and region
* set cluster region during onboarding for managed clusters
* wip
* fix for onboarding script
* add sp support for the login
* update help
* add sp support for powershell
* script updates for sp login
* wip
* wip
* wip
* readme updates
* update the links to use ci_prod branch
* fix links
* fix image link
* some more readme updates
* add missing serviceprincipal in ps scripts (#435)
* fix telemetry bug (#436)
* Gangams/readmeupdates non aks 09162020 (#437)
* changes for ciprod09162020 non-aks release
* fix script to handle cross sub scenario
* fix minor comment
* fix date in version file
* fix pr comments
* Gangams/fix weird conflicts (#439)
* separate build yamls for ci_prod branch (#415) (#416)
* [Merge] dev to prod for ciprod08072020 release (#424)
* separate build yamls for ci_prod branch (#415)
* re-enable adx path (#420)
* Gangams/release changes (#419)
* updates related to release
* updates related to release
* fix the incorrect version
* fix pr feedback
* fix some typos in the release notes
* fix for zero filled metrics (#423)
* consolidate windows agent image docker files (#422)
* consolidate windows agent image docker files
* revert docker file consolidation
* revert readme updates
* merge back windows dockerfiles
* image tag update
Co-authored-by: Vishwanath <visnara@microsoft.com>
Co-authored-by: rashmichandrashekar <rashmy@microsoft.com>
Co-authored-by: Vishwanath <visnara@microsoft.com>
Co-authored-by: rashmichandrashekar <rashmy@microsoft.com>
* fix quote issue for the region (#441)
* fix cpucapacity/limit bug (#442)
* grwehner/pv-usage-metrics (#431)
- Send persistent volume usage and capacity metrics to LA for PVs with PVCs at the pod level; config to include or exclude kube-system namespace.
- Send PV usage percentage to MDM if over the configurable threshold.
- Add PV usage recommended alert template.
* add new custom metric regions (#444)
* add new custom metric regions
* fix commas
* add 'Terminating' state (#443)
* Gangams/sept agent release tasks (#445)
* turnoff mdm nonsupported cluster types
* enable validation of server cert for ai ruby http client
* add kubelet operations total and total error metrics
* node selector label change
* label update
* wip
* wip
* wip
* revert quotes
* grwehner/pv-collect-volume-name (#448)
Collect and send the volume name as another tag for pvUsedBytes in InsightsMetrics, so that it can be displayed in the workload workbook. Does not affect the PV MDM metric
* Changes for september agent release (#449)
Moving from v1beta1 to v1 for health CRD
Adding timer for zero filling
Adding zero filling for PV metrics
* Gangams/arc k8s related scripts, charts and doc updates (#450)
* checksum annotations
* script update for chart from mcr
* chart updates
* update chart version to match with chart release
* script updates
* latest chart updates
* version updates for chart release
* script updates
* script updates
* doc updates
* doc updates
* update comments
* fix bug in ps script
* fix bug in ps script
* minor update
* release process updates
* use consistent name across scripts
* use consistent names
* Install CA certs from wireserver (#451)
* grwehner/pv-volume-name-in-mdm (#452)
Add volume name for PV to mdm dimensions and zero fill it
* Release changes for 10052020 release (#453)
* Release changes for 10052020 release
* remove redundant kubelet metrics as part of PR feedback
* Update onboarding_instructions.md (#456)
* Update onboarding_instructions.md
Updated the documentation to reflect where to update the config map.
* Update onboarding_instructions.md
* Update onboarding_instructions.md
* Update onboarding_instructions.md
Updated the link
* chart update for sept2020 release (#457)
* add missing version update in the script (#458)
* November release fixes - activate one agent, adx schema v2, win perf issue, syslog deactivation (#459)
* activate one agent, adx schema v2, win perf issue, syslog deactivation
* update chart
* remove hiphen for params in chart (#462)
Merging as its a simple fix (remove hiphen)
* Changes for cutting a new build for ciprod10272020 release (#460)
* using latest stable version of msys2 (#465)
* fixing the windows-perf-dups (#466)
* chart updates related to new microsoft/charts repo (#467)
* Changes for creating 11092020 release (#468)
* MDM exception aggregation (#470)
* grwehner/mdm custom metric regions (#471)
Remove custom metrics region check for public cloud
* updaitng rs limit to 1gb (#474)
* grwehner/pv inventory (#455)
Add fluentd plugin to request persistent volume info from the kubernetes api and send to LA
* Gangams/fix for build release pipeline issue (#476)
* use isolated cdpx acr
* correct comment
* add pv fluentd plugin config to helm rs config (#477)
* add pv fluentd plugin to helm rs config
* helm rbac permissions for pv api calls
* Gangams/fix rs ooming (#473)
* optimize kpi
* optimize kube node inventory
* add flags for events, deployments and hpa
* have separate function parseNodeLimits
* refactor code
* fix crash
* fix bug with service name
* fix bugs related to get service name
* update oom fix test agent
* debug logs
* fix service label issue
* update to latest agent and enable ephemeral annotation
* change stream size to 200 from 250
* update yaml
* adjust chunksizes
* add ruby gc env
* yaml changes for cioomtest11282020-3
* telemetry to track pods latency
* service count telemetry
* rename variables
* wip
* nodes inventory telemetry
* configmap changes
* add emit streams in configmap
* yaml updates
* fix copy and paste bug
* add todo comments
* fix node latency telemetry bug
* update yaml with latest test image
* fix bug
* upping rs memory change
* fix mdm bug with final emit stream
* update to latest image
* fix pr feedback
* fix pr feedback
* rename health config to agent config
* fix max allowed hpa chunk size
* update to use 1k pod chunk since validated on 1.18+
* remove debug logs
* minor updates
* move defaults to common place
* chart updates
* final oomfix agent
* update to use prod image so that can be validated with build pipeline
* fix typo in comment
* Gangams/enable arc onboarding to ff (#478)
* wip
* updates
* trigger login if the ctx cloud not same as specified cloud
* add missed commit
* Convert PV type dictionary to json for telemetry so it shows up in logs (#480)
* fix 2 windows tasks - 1) Dont log to termination log 2) enable ADX route for containerlogs in windows (for O365) (#482)
* fix ci envvar collection in large pods (#483)
* grwehner/jan agent tasks (#481)
- Windows agent fix to use log filtering settings in config map.
- Error handling for kubelet_utils get_node_capacity in case /metrics/cadvsior endpoint fails.
- Remove env variable for workspace key for windows agent
* updating fbit version and cpu limit (#485)
* reverting to older version (#487)
* Gangams/add fbsettings configurable via configmap (#486)
* wip
* fbit config settings
* add config warn message
* handle one config provided but not other
* fixed pr feedback
* fix copy paste error
* rename config parameter names
* fix typo
* fix fbit crash in helm path
* fix nil check
* Gangams/jan agent release tasks (#484)
* wip
* explicit amd64 affinity for hybrid workloads
* fix space issue
* wip
* revert vscode setting file
* remove per container logs in ci (#488)
* updates for ciprod01112021 release (#489)
* new yaml files (#491)
* Use cloud-specific instrumentation keys (#494)
If APPLICATIONINSIGHTS_AUTH_URL is set/non-empty then the agent will now grab a custom IKey from a URL stored in APPLICATIONINSIGHTS_AUTH_URL
* upgrade apt to latest version (#492)
* upgrade apt to latest version
* fix pr feedback
* Gangams/add support for extension msi for arc k8s cluster (#495)
* wip
* add env var for the arc k8s extension name
* chart update
* extension msi updates
* fix bug
* revert chart and image to prod version
* minor text changes
* image tag to prod
* wip
* wip
* wip
* wip
* final updates
* fix whitespaces
* simplify crd yaml
* Gangams/arm template arc k8s extension (#496)
* arm templates for arc k8s extension
* update to use official extension type name
* update
* add identity property
* add proxyendpointurl parameter
* add default values
* Gangams/aks monitoring via policy (#497)
* enable monitoring through policy
* wip
* handle tags
* wip
* add alias
* wip
* working
* updates
* working
* with deployment name
* doc updates
* doc updates
* fix typo in the docs
* revert to use operatingSystem from osImage for node os telemety (#498)
* Container log v2 schema changes (#499)
* make pod name in mdsd definition as str for consistency. msgp has no type checking, as it has type metadata in it the message itself.
* Add priority class to the daemonsets (#500)
* Add priority class to the daemonsets
Add a priority class for omsagent and have the daemonsets use this
to be sure to schedule the pods.
Daemonset pods are constrained in scheduling to run on specific
nodes. This is done by the daemonset controller. When a node shows
up it will create a pod with a strong affinity to that node. When a
node goes away, it will delete the pod with the node affinity to that
node.
Kubernetes pod scheduling does not know it is a daemonset but it does
know it is tied to a specific node. With default scheduling, it is
possible for the pods to be "frozen out" of a node because the node
already is full. This can happen because "normal" pods may already
exist and are looking for a node to get scheduled on when a node is
added to the cluster. The daemonset controller will only first
create the pod for the node at around the same time. The kubernetes
scheduler is running async from all of this and thus there can be a
race as to who gets scheduled on the node.
The pod priority class (and thus the pod priority) is a way to indicate
that the pod has a higher scheduling priority than a default pod.
By default, all pods are at priority 0. Higher numbers are higher
priority. Setting the priority to something greater than zero will
allow the omsagent daemonsets to win a race against "normal" pods for
scheduled resources on a node - and will also allow for graceful
eviction in the case the node is too full.
Without this, omsagent can be left out of node in clusters that are
very busy, especially in dynamic scaling situations.
I did not test the windows pod as we have no windows clusters.
* CR feedback
* fix node metric issue (#502)
* Bug fixes for Feb release (#504)
* bug fix for mdm metrics with no limits
* fix exception bug
* Gangams/feb 2021 agent bug fix (#505)
* fix npe in getKubeServiceRecords
* use image fields from spec
* fix typo
* cover all cases
* handle scenario only digest specified
* changes for release -ciprod02232021 (#506)
* Gangams/e2e test framework (#503)
* add agent e2e fw and tests
* doc and script updates
* add validation script
* doc updates
* yaml updates
* fix typo
* doc updates
* more doc updates
* add ISTEST for helm chart to use arc conf
* refactor test code
* fix pr feedback
* fix pr feedback
* fix pr feedback
* fix pr feedback
* scrape new kubelet pod count metric name (#508)
* Adding explicit json output to az commands as the script fails if az is configured with Table output #409 (#513)
* Gangams/arc proxy contract and token renewal updates (#511)
* fix issue with crd status updates
* handle renewal token delays
* add proxy contract
* updates for proxy cert for linux
* remove proxycert related changes
* fix whitespace issue
* fix whitespace issue
* remove proxy in arm template
* doc updates for microsoft charts repo release (#512)
* doc updates for microsoft charts repo release
* wip
* Update enable-monitoring.sh (#514)
Line 314 and 343 seems to have trailing spaces for some subscriptions which is exiting the script even for valid scenarios
Co-authored-by: Ganga Mahesh Siddem <gangams@microsoft.com>
* Prometheus scraping from sidecar and OSM changes (#515)
* add liveness timeout for exec (#518)
* chart and other updates (#519)
* Saaror osmdoc (#523)
* Create ReadMe.md
* Update ReadMe.md
* Update ReadMe.md
* Update ReadMe.md
* Update ReadMe.md
* Add files via upload
* Update ReadMe.md
* Update ReadMe.md
* Update ReadMe.md
* Update ReadMe.md
* Update ReadMe.md
* Update ReadMe.md
* telemetry bug fix (#527)
* Fix conflicting logrotate settings (#526)
The node and the omsagent container both have a cron.daily file to rotate certain logs daily. These settings are the same for some files in /var/log (mounted from the node with read/write access), causing the rotation to fail when both try to rotate at the same time. So then the /var/log/*.1 file is written to forever. Since these files are always written to and never rotated, it causes high memory usage on the node after a while.
This fix removes the container logrotate settings for /var/log, which the container does not write to.
* bug fix (#528)
* Gangams/arc ev2 deployment (#522)
* ev2 deployment for arc k8s extension
* fix charts path issue
* rename scripts tar
* add notifications
* fix line endings
* fix line endings
* update with prod repo
* fix file endings
* added liveness and telemetry for telegraf (#517)
* added liveness and telemetry for telegraf
* code transfer
* removed windows liveness probe
* done
* Windows metric fix (#530)
* changes
* about to remove container fix
* moved caching code to existing loop
* removed un-necessary changes
* removed a few more un-necessary changes
* added windows node check
* fixed a bug
* everything works confirmed
* OSM doc update (#533)
* Adding MDM metrics for threshold violation (#531)
* Rashmi/april agent 2021 (#538)
* add Read_from_Head config for all fluentbit tail plugins (#539)
See the commit message of: fluent/fluent-bit@70e33fa
for details explaining the fluentbit change and what Read_from_Head does when set to true.
* fix programdata mount issue on containerd win nodes (#542)
* Update sidecar mem limits (#541)
* David/release 4 22 2021 (#544)
* updating image tag and agent version
* updated liveness probe
* updated release notes again
* fixed date in version file
* 1m, 1m, 1s by default (#543)
* 1m, 1m, 1s by default
* setting default through a different method
* David/aad stage 1 release (#556)
* update to latest omsagent, add eastus2 to mdsd regions
* copied oneagent bits to a CI repository release
* mdsd inmem mode
* yaml for cl scale test
* yaml for cl scale test
* reverting dockerProviderVersion version to 15.0.0
* prepping for release (updated image version, dockerProviderVersion, and release notes
* container log scaletest yamls
* forgot to update image version in chart
* fixing windows tag in dockerfile, changing release notes wording
* missed windows tag in one more place
* forgot to change the windows dockerProviderVersion back
Co-authored-by: Ganga Mahesh Siddem <gangams@microsoft.com>
* Update ReleaseNotes.md (#558)
fix imagetag in the release notes
* Add wait time for telegraf and also force mdm egress to use tls 1.2 (#560)
* Add wait time for telegraf and also force mdm egress to use tls 1.2
* add wait for all telegraf dependencies across all containers (ds & rs)
* remove ssl change so we dont include as part of the other fix until we test with att nodes.
* partially disabled telegraf liveness probe check, we'll still have telemetry but the probe won't fail if telegraf isn't running (#561)
* changes for 05202021 release (#563)
* changes for 05202021 release
* fixed typos
* Rashmi/jedi wireserver (#566)
* Update ReadMe.md (#565)
* Update ReadMe.md
* Update ReadMe.md
Included feedback from OSM team and Fixed
* Gangams/aad stage2 full switch to mdsd (#559)
* full switch to mdsd, upgrade to ruby v1 & omsagent removal
* add odsdirect as fallback option
* cleanup
* cleanup
* move customRegion to stage3
* updates related to containerlog route
* make xml eventschema consistent
* add buffer settings
* address HTTPServerException deprecation in ruby 2.6
* update to official mdsd version
* fix log message issue
* fix pr feedback
* get ridoff unused code from omscommon
* fix pr feedback
* fix pr feedback
* clean up
* clean up
* fix missing conf
* Send perf metrics to MDM from windows daemonset (#568)
* updating json gem to address CVE-2020-10663 (#567)
* updating json gem to address CVE-2020-10663
* updating json gem to address CVE-2020-10663
* update recommended alerts readme (#570)
@dcbrown16 pointed out that this page links to the wrong document in [this issue](https://github.com/microsoft/Docker-Provider/issues/475). The content in the currently linked page is identitical to the page which should be linked, so it's a simple fix.
* trying again to fix the json gem (#571)
* trying again to fix the json gem
* removing installation of newer json gem
* Addressing PR comments for - https://github.com/microsoft/Docker-Provider/pull/568 (#569)
* Mem_Buf_limit is configurable via ConfigMap (#574)
* add log rotation settings for fluentd logs (#577)
* Gangams/release 06112021 (#578)
* updates related to ciprod06112021 release
* minor update
* release note update (#579)
* Make sidecar fluentbit chunk size configurable (#573)
* Fix vulnerabilities (#583)
* test
* test1
* test-2
* test-3
* 3
* 4
* test
* 2
* 3
* 4
* 5
* 6
* rename gem for windows
* fix
* fix
* Windows build optimization (#582)
* fix windows build failure due to msys2 version
* Fix telegraf startup issue when endpoint is unreachable (#587)
* revert fbit tail plugins defaults to std defaults (#586)
* fixed another bug (#593)
* feat: add new metrics to MDM for allocatable % calculation of cpu and memory usage (#584)
* feat: allocatable cpu and memory % metrics for MDM
* maybe
* linux is working
* windwos....
* some more
* comment
* better
* syntax
* ruby
* revert omsagent.yaml
* comments
* pr feedback
* pr feedback
* testing msys2 version update
* better
* update adx sdk for perf issue (#601)
* remove md check
* Gangams/release notes update for hotfix (#596)
* release notes updates
* release notes updates for ciprod06112021-1
* Cherry picking hotfix changes to ci_dev (#605)
* release changes (#607)
* Gangams/aad stage3 msi auth (#585)
* changes related to aad msi auth feature
* use existing envvars
* fix imds token expiry interval
* refactor the windows agent ingestion token code
* code cleanup
* fix build errors
* code clean up
* code clean up
* code clean up
* code clean up
* more refactoring
* fix bug
* fix bug
* add debug logs
* add nil checks
* revert changes
* revert yaml change since this added in aks side
* fix pr feedback
* fix pr feedback
* refine retry code
* update mdsd env as per official build
* cleanup
* update env vars per mdsd
* update with mdsd official build
* skip cert gen & renewal incase of aad msi auth
* add nil check
* cherry windows agent nodeip issue
* fix merge issue
Co-authored-by: rashmichandrashekar <rashmy@microsoft.com>
* Gangams/remove chart version dependency (#589)
* remove chart version dependency
* remove unused code
* fix resource type
* fix
* handle weird cli chars
* update release process
* Gangams/july 2021 release tasks 3 (#613)
* use artifact and pipeline creds for image push
* minor update
* add vuln fix here so that pr can be merged
* remove un-used output plugin (#614)
* fix telegraf telemetry and improve fluentd liveness (#611)
* fix telegraf telemetry and improve fluentd liveness
* address identified vuln with libsystemd0
* fix exported image file extension
* Gangams/july 2021 release tasks 2 (#612)
* tail rs mdsd err logs
* configure mdsd log rotation
* log rotation for mdsd log files
* Fix out_oms.go dependency vulnerabilities (#623)
* revert libsystemd0 update (#616)
* updates for ci-prod release instructions (#619)
* cherry pick changes from ci_prod (#622)
* Support az login for passwords starting with dash ('-') (#626)
Co-authored-by: Vladimir Babichev <vladimir.babichev@finastra.com>
* Gangams/add telemetry fbit settings (#628)
* add telemetry to track fbit settings
* add telemetry to track fbit settings
* check onboarding status (#629)
* Gangams/arc k8s conformance test updates (#617)
* conf test updates
* clean up
* wip
* update with mcr cidev image
* handle log path
* cleanup
* clean up
* wip
* working
* update for mcr image
* minor
* image update
* handle latency of connected cluster resource creation
* update conftest image
* upgrade golang version for windows in pipeline build and locally (#630)
* Updating a link in Readme.md (#632)
The link to the build pipelines now goes directly to our build pipelines (instead of to all github-private pipelines)
* Updating omsagent yaml to have parity with omsagent yaml file in AKS RP (#615)
* Unit test tooling (#625)
Added tooling and examples for unit tests
* run unit tests after a merge too (#634)
* flag stale PRs & issues
* Adding script to collect logs (for troubleshooting) (#636)
* added script for collecting logs
* added windows daemonset and prometheus sidecar, as well as some explanatory prints
* added kubectl describe and kubectl logs output
* changed message to make it more clear some erros are expected
* Sarah/ev2 (#640)
* ev2 artifacts for release pipeline
* update parameters reference
* add artifacts tar file
* changes to rollout and service model
* change agentimage path
* adding agentimage to artifact script
* removing charts from tarball
* change script to use blob storage
* change blob variables
* echo variables
* change blob uri
* use release id for blob prefix
* change to delete blob file
* add check for if blob storage file exists
* fix script errors
* update check for file in storage
* change true check
* comments and change storage account info to pipeline variables
* Changes for windows tar file
* PR changes
* documenting fbit tail plugin configmap settings. (#638)
* documenting fbit tail plugin configmap settings.
* Install unzip package on shell extension (#642)
* Changing installation in ev2 script (#644)
* Adjust release pipeline to use cdpx acr (#647)
* Adjust release pipeline to use cdpx acr
* Adjust release pipeline to use cdpx acr
* Update CDPX ACR path
* Add check for cdpx repo variable
* Sarah/ev2 prod (#649)
* Ev2 changes for prod
* CDPX repo naming change (#652)
* Sarah/ev2 update (#654)
* remove acr name from repo path
* add check to make sure tag does not exist in mcr repo
* change tag syntax for mcr repo check (#655)
* Gangams/optimize win livenessprobe (#653)
* livenessprobe optimization
* optimize windows agent liveness probe
* optimize windows agent liveness probe
* optimize windows agent liveness probe
* optimize windows agent liveness probe
* optimize windows agent liveness probe
* optimize windows agent liveness probe
* optimize windows agent liveness probe
* optimize windows agent liveness probe
* Gangams/addon token adapter image tag to telemetry (#656)
* addon token adapter image tag
* addon token adapter image tag
* Sarah/ev2 helm (#658)
* Use MSI for Arc Release
* Use CIPROD_ACR AME subscription for shell extension
* remove extra line endings
* Sarah/ev2 pipeline (#661)
* testing build artifact dir changes
* add .pipelines directory and omsagent.yaml to build artifacts
* add charts directory to build artifacts (#662)
* Sarah/remove cdpx creds (#664)
* don't use cdpx acr creds from kv
* add e2etest.yaml to build output
* keep cdpx creds for now
* chart updates for rbac api version change (#660)
* chart updates for rbac api version change
* include windows ds for arc
* proxy support (for non-aks) (#665)
* changes related to aad msi auth feature
* use existing envvars
* fix imds token expiry interval
* initial proxy support
* merge?
* cleaning up some files which should've merged differently
* proxy should be working, but most tables don't have any data. About to merge, maybe whatever was wrong is now fixed
* linux AMA proxy works
* about to merge
* proxy support appears to be working, final mdsd build location will still change
* removing some unnecessary changes
* forgot to remove one last change
* redirected mdsd stderr to stdout instead of stdin
* addressing proxy password location comment
Co-authored-by: Ganga Mahesh Siddem <gangams@microsoft.com>
* Gangams/agent release ciprod10082021 & win-ciprod10082021 (#666)
* updates for the release ciprod10082021 and win-ciprod10082021
* updates for the release ciprod10082021 and win-ciprod10082021
* updates for the release ciprod10082021 and win-ciprod10082021
* updates for the release ciprod10082021 and win-ciprod10082021
* use buildcommand for prod pipeline (#668)
* fixed merge issues. (#671) (#672)
* fix merge conflicts
* update with newimage tag
* changes related to mdsd version update (#673) (#674)
* Sarah/enable metrics (#675)
* add user assigned msi to yaml for pipeline
* update placeholders
* Gangams/chart updates oct2021 release (#676)
* chart updates for oct2021 release
* wip
* wip
* wip
* Gangams/msi mode mdsd crash fix (#677)
* update mdsd version which has fix for crash in msi mode
* image tag updates
* update to use extension GA api version (#679)
* Gangams/arm template msi onboarding (#659)
* wip
* wip
* working
* working
* working
* working
* working
* working
* shorten dcr prefix to DCR- to handle default workspace name length
* use MSCI- prefix similar to MSVMI- for dcr
* Gangams/conf test updates to handle sidecar (#681)
* wip
* test updates
* fix pr feedback
* fix pr feedback
* Fix scan break due to latest trivy changes
* Anjohans/configurable database name (#663)
* First cut at an implementation
* Reverting a change
* Moving a few lines to better align with cluster URI config
* Moving a few lines to better align with cluster URI config
* Adding an extra check that won't hurt
* Getting ADX database name from config rather than from secret
* Reverse the mangling done by editor
* Fixes to the code for reading the db name setting
* More fixes to the rb code for settings
* Tweaked and tested
* Code review
* Review follow-up
* Remove whitespace
* Gangams/troubelshooting script for arc k8s (#682)
* wip
* wip
* wip
* wip
* wip
* wip
* wip
* wip
* wip
* wip
* wip
* wip
* wip
* wip
* wip
* wip
* wip
* wip
* doc updates
* doc updates
* wip
* wip
* update repo for issues
* fix minor one
* Sarah/remove cdpx creds (#685)
* remove download of cdpx creds
* fix: subtract number instead of string + update fluentd version 1.14.2 to fix security vulnerability (#686)
* fix: change default value to a number so that substraction happens correctly
* update fluentd version to 1.14.2
* extra end statement
* safely set to float
* big decimal precision
* revert omsagent
* keep telemetry
* Faster Linux builds (part 1) (#687)
* moved docker image arg later on to enable docker build caching
* fixing image tag (doh)
* Sarah/fluentbit windows log (#688)
* upgrade fluentbit version for windows
* saving progress--fluent bit log tailing working for windows
* use configmap values for fluent-bit.conf where necessary and make necessary files common
* revert certificategenerator
* remove tomlparser-agent-config from linux folder
* clean up fluent.conf
* clean up fluent-bit.conf
* revert image tag
* fix agent tag
* make fluent bit flush interval configurable
* clean up unecessary conf files
* remove unecessary parts of fluent and fluent-bit conf
* log level back to info
* add fbit env variables for omsagent-win
* moving db files to var directory
* default to port 10250 & containerd for linux agent (#699)
* default to port 10250 & containerd
* fix pr feedback
* Updating pod annotation for latest agent version (#697)
* fix windows build failure due to msys2 version (#700)
* fix windows build failure due to msys2 version
* 20211130.0.0
* Jan agent tasks (#698)
* remove v1 fallback hidden option (#705)
* collect telemetry containerlog records with emptystamp (#703)
* collect telemetry containerlog records with emptystamp
* collect telemetry containerlog records with emptystamp
* Fixing telegraf bug for placeholder name (#706)
* Gangams/jan 2022 release tasks 3 (#702)
* add telemetry related to windows containers records
* add telemetry related to windows containers records
* containercount telemetry
* add explicit exit code in ps scripts
* node count telemetry
* telemetry for win cirecord 64KB or more
* metric to track wintelegraf metrics with tags 64kb
* metric to track wintelegraf metrics with tags 64kb
* fix pr feedback
* Gangams/jan 2022 release tasks 2 (#701)
* mdsd proc cpu and memory telemetry
* write ai logs to file and telemetry for mdsd proc
* write ai logs to file and telemetry for mdsd proc
* write ai logs to file and telemetry for mdsd proc
* fix pr feedback
* use name_prefix
* remove mdsd telemetry changes
* remove mdsd telemetry changes
* remove mdsd telemetry changes
* release updates for ciprod01312022 & win-ciprod01312022release (#707)
* release updates for ciprod01312022 release
* release updates for ciprod01312022 release
* fix pr feedback
* fix logger exception (#709)
* Gangams/chart version update for jan release (#710)
* chart updates for jan2022 release
* add missing agentversion annotations
* fix agentversion annotation issue in chart (#712)
* adx bug + misc (#714)
* fix golang dependencies
* fix adx bug
* exclude telegraf
* fix space
* include both
* exclude files specifically
* fix build break (#715)
* fix build break
* update all places
* Explicitly use win-2019 to unblock windows PRs builds
* Fixing telegraf vulnerability (#716)
* cherry picked changes from 03112022 release (#719)
* cherry picked changes from 03112022 release
* Gangams/http proxy support (#717)
* add proxy cert support
* add proxy cert support
* add proxy cert support
* add proxy cert support
* remove arbitery username and pwd requirement
* remove arbitery username and pwd requirement
* add proxy support for mdm
* mdsd dev build
* proxy changes
* fix typo
* mdsd dev build
* add libcurl specific things
* working mdsd proxy build
* mdsd official master build
* handle proxy endpoint which endswith /
* latest official mdsd build
* add telemetry to track proxy ca cert
* build multi-arch images (#704)
* build multi-arch linux images
* new pipelines to build multi-arch images
Co-authored-by: Amol Agrawal <amagraw@microsoft.com>
* add missing artifacts (#720)
* add missing artifacts
Co-authored-by: Amol Agrawal <amagraw@microsoft.com>
* Gangams/msi onboarding arm template updates for AKS (#721)
* msi arm template updates
* handle space in location
* minor fixes (#722)
Co-authored-by: Amol Agrawal <amagraw@microsoft.com>
* specify go patch version (#723)
* specify go minor version
Co-authored-by: Amol Agrawal <amagraw@microsoft.com>
* User/amagraw/ciprod release 20220317 (#724)
* ciprod release march changes
Co-authored-by: Amol Agrawal <amagraw@microsoft.com>
* Remove health type from DCR onboarding & add private link support for windows agent in msi mode (#727)
* add private link support for windows agent in msi auth
* remove Microsoft-KubeHealth
* add private link support for windows msi
* fix bug
* fix bug
* fix bug
* fix bug
* check platform specific tags (#730) (#731)
* PodReadyPercentage metric bug fix (#734)
* update windows to ruby 2.7 (#732)
Co-authored-by: Amol Agrawal <amagraw@microsoft.com>
* Improve CI/CD for multi-arch (#733)
* selective push + trivy test
* keep size down
* improve CI and PR builds
* improve checks
* remove IMAGE_TAG build_arg from prod pipeline
Co-authored-by: Amol Agrawal <amagraw@microsoft.com>
* Gangams/ts updates for msi (#736)
* ts updates for msi based onboarding
* ts updates for msi based onboarding
* fix typo
* fix typo
* improve log message
* Sarah/health deprecation (#735)
Removes all health feature related code
* check platform specific tags (#738)
Co-authored-by: Amol Agrawal <amagraw@microsoft.com>
* Gangams/msi test instructions (#739)
* instructions for msi test validation
* readme updates
* readme updates
* readme updates
* readme updates
* Add CI Windows Build to MultiArch Dev pipeline (#740)
* test image in pools
* update dev pipeline - 1
* update dev -1
* fix job names
* correct paths
* test pool name
* update pool name
* updated urls
* speed up installs
* add base build
* fix paths
* do both builds
* fix bug
* add pool for common
* fix bug
* create path
* temp remove metadata windows
* fix bug
* fix docker command
* almost there
* login to acr
* create windows metadata file
* address PR comments I
Co-authored-by: Amol Agrawal <amagraw@microsoft.com>
* Add Windows phase (#741)
* build and release windows for prod
Co-authored-by: Amol Agrawal <amagraw@microsoft.com>
* Sarah/add onboarding templates (#742)
* add onboarding templates for legacy auth
* fix download (#749)
Co-authored-by: Amol Agrawal <amagraw@microsoft.com>
* force run trivy stage (#745)
- scans for HIGH, MEDIUM, CRITICAL CVEs with fixes available in / and /usr/lib
- breaks build if CVEs with existing fixes found
- adds trivyignore to accomodate CVEs which are understood and should not get flagged
- adds CVEs to trivyignore to unblock builds; CVEs will be fixed and removed from trivyignore in later PRs
Co-authored-by: Amol Agrawal <amagraw@microsoft.com>
* update telegraf to 1.22.2 to fix vulns (#752)
* update telegraf to 1.22.2 to fix vulns
Co-authored-by: Amol Agrawal <amagraw@microsoft.com>
* Gangams/arc k8s aad msi auth (#743)
* arc k8s msi
* wip
* extension identity role
* imds sidecar integration for arc k8s
* imds sidecar integration for arc k8s
* imds endpoint for windows
* imds endpoint for windows
* wip
* fix exception
* rename param name
* arc msi imdsd container changes
* arc msi imdsd container changes
* arc msi imdsd container changes
* arc msi imdsd container changes
* arc msi imdsd container changes
* revert unneeded yaml changes
* revert unneeded yaml changes
* wip
* wip
* working
* working
* working
* add implementation for msi token for windows mdm metrics
* fix comment
* arc k8s msi onboarding templates
* fix template bug
* fix template bug
* fix template bug
* rename flag name
* fix template bug
* make useAADAuth specific to arc k8s
* set k8sport at machine scope for windows
* fix bug
* fix bug
* update rbac for arc k8s imds
* bump chart version for conformance test run
* conf test updates for msi auth
* cli extension whl file
* add containerinsights solution in msi auth mode
* unify tags
* revert test chart and image versions
* remove test whl file and fix conf test
* conf test updates for addon-token-adapter
* remove container insights solution add for msi auth
* add missing arm template param
* Gangams/ws2022 support (#756)
* use hyperv isolation
* multi-arc image support
* multi-arc image support
* multi-arc image support
* multi-arc image support
* multi-arc image support
* multi-arc image support
* multi-arc image support
* multi-arc image support
* multi-arc image support
* multi-arc image support
* doc and script updates
* add common as dependency for multi-arc job
* merge into single job for perf evaluation
* merge into single job for perf evaluation
* merge into single job for perf evaluation
* separate jobs for ltsc2019 & ltsc2022
* separate jobs for ltsc2019 & ltsc2022
* update dev image docker file & script
* remove unnecessary task
* update prod pipeline yaml for windows multi-arc image
* test yamls for ltsc2019 & ltsc2022
* fix pr checker fail
* fix repoImageWindows path in windows pipeline
* remove passing imagetag for prod
* CA Cert Fix for Mariner Hosts in Air Gap (#751)
* add cifs & fuse file systems to ignore list (#750)
* Data collection script (#759)
* Add files via upload
* Add files via upload
* Delete AKSInsightsLogCollection.sh
* Create README.md
* Add files via upload
* move script to subfolder LogCollection
* Update README.md
* Rename AKSInsightsLogCollection.sh to AgentLogCollection.sh
* Microsoft mandatory file (#763)
Co-authored-by: microsoft-github-policy-service[bot] <77245923+microsoft-github-policy-service[bot]@users.noreply.github.com>
* Adding v2 schema options (#762)
* Adding v2 schema options
Adding commented out section in log collection settings for v2 schema
* adding documentation link
* Agent release for ciprod05192022 and win-ciprod05192022 (#765)
* Making changes for the release ciprod05192022 (except release notes)
* Adding release notes
* Remove unnecessary spaces
* Updating release notes for configmap v2 and disk usage metrics
Co-authored-by: Ganga Mahesh Siddem <gangams@microsoft.com>
Co-authored-by: Vishwanath <visnara@microsoft.com>
Co-authored-by: rashmichandrashekar <rashmy@microsoft.com>
Co-authored-by: bragi92 <kadubey@microsoft.com>
Co-authored-by: saaror <31900410+saaror@users.noreply.github.com>
Co-authored-by: Grace Wehner <grace.wehner@microsoft.com>
Co-authored-by: deagraw <deagraw@microsoft.com>
Co-authored-by: David Michelman <daweim0@gmail.com>
Co-authored-by: Michael Sinz <36865706+Michael-Sinz@users.noreply.github.com>
Co-authored-by: Nicolas Yuen <nicolas.yuen@gmail.com>
Co-authored-by: seenu433 <seenu433@gmail.com>
Co-authored-by: Tsubasa Nomura <tsubasa@nomupro.com>
Co-authored-by: Vladimir <vladimir.d.babichev@gmail.com>
Co-authored-by: Vladimir Babichev <vladimir.babichev@finastra.com>
Co-authored-by: sarahpeiffer <46665092+sarahpeiffer@users.noreply.github.com>
Co-authored-by: Anders Johansen <anjohans@microsoft.com>
Co-authored-by: Amol Agrawal <pfrcks@gmail.com>
Co-authored-by: Amol Agrawal <amagraw@microsoft.com>
Co-authored-by: Nina <47805999+MSNina123456@users.noreply.github.com>
Co-authored-by: microsoft-github-policy-service[bot] <77245923+microsoft-github-policy-service[bot]@users.noreply.github.com>
Co-authored-by: Auston Li <al884@cornell.edu>
* separate build yamls for ci_prod branch (#415)
* re-enable adx path (#420)
* Gangams/release changes (#419)
* updates related to release
* updates related to release
* fix the incorrect version
* fix pr feedback
* fix some typos in the release notes
* fix for zero filled metrics (#423)
* consolidate windows agent image docker files (#422)
* consolidate windows agent image docker files
* revert docker file consolidation
* revert readme updates
* merge back windows dockerfiles
* image tag update
* Gangams/cluster creation scripts (#414)
* onprem k8s script
* script updates
* scripts for creating non-aks clusters
* fix minor text update
* updates
* script updates
* fix
* script updates
* fix scripts to install docker
* fix: Pin to a particular version of ltsc2019 by SHA (#427)
* enable collecting npm metrics (optionally) (#425)
* enable collecting npm metrics (optionally)
* fix default enrichment value
* fix adx
* Saaror patch 3 (#426)
* Create README.MD
Creating content for Kubecon lab
* Update README.MD
* Update README.MD
* Gangams/add containerd support to windows agent (#428)
* wip
* wip
* wip
* wip
* bug fix related to uri
* wip
* wip
* fix bug with ignore cert validation
* logic to ignore cert validation
* minor
* fix minor debug log issue
* improve log message
* debug message
* fix bug with nullorempty check
* remove debug statements
* refactor parsers
* add debug message
* clean up
* chart updates
* fix formatting issues
* Gangams/arc k8s metrics (#413)
* cluster identity token
* wip
* fix exception
* fix exceptions
* fix exception
* fix bug
* fix bug
* minor update
* refactor the code
* more refactoring
* fix bug
* typo fix
* fix typo
* wait for 1min after token renewal request
* add proxy support for arc k8s mdm endpoint
* avoid additional get call
* minor line ending fix
* wip
* have separate log for arc k8s cluster identity
* fix bug on creating crd resource
* remove update permission since not required
* fixed some bugs
* fix pr feedback
* remove list since its not required
* fix: Reverting back to ltsc2019 tag (#429)
* more kubelet metrics (#430)
* more kubelet metrics
* celan up new config
* fix nom issue when config is empty (#432)
* support multiple docker paths when docker root is updated thru knode (#433)
* Gangams/doc and other related updates (#434)
* bring back nodeslector changes for windows agent ds
* readme updates
* chart updates for azure cluster resourceid and region
* set cluster region during onboarding for managed clusters
* wip
* fix for onboarding script
* add sp support for the login
* update help
* add sp support for powershell
* script updates for sp login
* wip
* wip
* wip
* readme updates
* update the links to use ci_prod branch
* fix links
* fix image link
* some more readme updates
* add missing serviceprincipal in ps scripts (#435)
* fix telemetry bug (#436)
* Gangams/readmeupdates non aks 09162020 (#437)
* changes for ciprod09162020 non-aks release
* fix script to handle cross sub scenario
* fix minor comment
* fix date in version file
* fix pr comments
* Gangams/fix weird conflicts (#439)
* separate build yamls for ci_prod branch (#415) (#416)
* [Merge] dev to prod for ciprod08072020 release (#424)
* separate build yamls for ci_prod branch (#415)
* re-enable adx path (#420)
* Gangams/release changes (#419)
* updates related to release
* updates related to release
* fix the incorrect version
* fix pr feedback
* fix some typos in the release notes
* fix for zero filled metrics (#423)
* consolidate windows agent image docker files (#422)
* consolidate windows agent image docker files
* revert docker file consolidation
* revert readme updates
* merge back windows dockerfiles
* image tag update
Co-authored-by: Vishwanath <visnara@microsoft.com>
Co-authored-by: rashmichandrashekar <rashmy@microsoft.com>
Co-authored-by: Vishwanath <visnara@microsoft.com>
Co-authored-by: rashmichandrashekar <rashmy@microsoft.com>
* fix quote issue for the region (#441)
* fix cpucapacity/limit bug (#442)
* grwehner/pv-usage-metrics (#431)
- Send persistent volume usage and capacity metrics to LA for PVs with PVCs at the pod level; config to include or exclude kube-system namespace.
- Send PV usage percentage to MDM if over the configurable threshold.
- Add PV usage recommended alert template.
* add new custom metric regions (#444)
* add new custom metric regions
* fix commas
* add 'Terminating' state (#443)
* Gangams/sept agent release tasks (#445)
* turnoff mdm nonsupported cluster types
* enable validation of server cert for ai ruby http client
* add kubelet operations total and total error metrics
* node selector label change
* label update
* wip
* wip
* wip
* revert quotes
* grwehner/pv-collect-volume-name (#448)
Collect and send the volume name as another tag for pvUsedBytes in InsightsMetrics, so that it can be displayed in the workload workbook. Does not affect the PV MDM metric
* Changes for september agent release (#449)
Moving from v1beta1 to v1 for health CRD
Adding timer for zero filling
Adding zero filling for PV metrics
* Gangams/arc k8s related scripts, charts and doc updates (#450)
* checksum annotations
* script update for chart from mcr
* chart updates
* update chart version to match with chart release
* script updates
* latest chart updates
* version updates for chart release
* script updates
* script updates
* doc updates
* doc updates
* update comments
* fix bug in ps script
* fix bug in ps script
* minor update
* release process updates
* use consistent name across scripts
* use consistent names
* Install CA certs from wireserver (#451)
* grwehner/pv-volume-name-in-mdm (#452)
Add volume name for PV to mdm dimensions and zero fill it
* Release changes for 10052020 release (#453)
* Release changes for 10052020 release
* remove redundant kubelet metrics as part of PR feedback
* Update onboarding_instructions.md (#456)
* Update onboarding_instructions.md
Updated the documentation to reflect where to update the config map.
* Update onboarding_instructions.md
* Update onboarding_instructions.md
* Update onboarding_instructions.md
Updated the link
* chart update for sept2020 release (#457)
* add missing version update in the script (#458)
* November release fixes - activate one agent, adx schema v2, win perf issue, syslog deactivation (#459)
* activate one agent, adx schema v2, win perf issue, syslog deactivation
* update chart
* remove hiphen for params in chart (#462)
Merging as its a simple fix (remove hiphen)
* Changes for cutting a new build for ciprod10272020 release (#460)
* using latest stable version of msys2 (#465)
* fixing the windows-perf-dups (#466)
* chart updates related to new microsoft/charts repo (#467)
* Changes for creating 11092020 release (#468)
* MDM exception aggregation (#470)
* grwehner/mdm custom metric regions (#471)
Remove custom metrics region check for public cloud
* updaitng rs limit to 1gb (#474)
* grwehner/pv inventory (#455)
Add fluentd plugin to request persistent volume info from the kubernetes api and send to LA
* Gangams/fix for build release pipeline issue (#476)
* use isolated cdpx acr
* correct comment
* add pv fluentd plugin config to helm rs config (#477)
* add pv fluentd plugin to helm rs config
* helm rbac permissions for pv api calls
* Gangams/fix rs ooming (#473)
* optimize kpi
* optimize kube node inventory
* add flags for events, deployments and hpa
* have separate function parseNodeLimits
* refactor code
* fix crash
* fix bug with service name
* fix bugs related to get service name
* update oom fix test agent
* debug logs
* fix service label issue
* update to latest agent and enable ephemeral annotation
* change stream size to 200 from 250
* update yaml
* adjust chunksizes
* add ruby gc env
* yaml changes for cioomtest11282020-3
* telemetry to track pods latency
* service count telemetry
* rename variables
* wip
* nodes inventory telemetry
* configmap changes
* add emit streams in configmap
* yaml updates
* fix copy and paste bug
* add todo comments
* fix node latency telemetry bug
* update yaml with latest test image
* fix bug
* upping rs memory change
* fix mdm bug with final emit stream
* update to latest image
* fix pr feedback
* fix pr feedback
* rename health config to agent config
* fix max allowed hpa chunk size
* update to use 1k pod chunk since validated on 1.18+
* remove debug logs
* minor updates
* move defaults to common place
* chart updates
* final oomfix agent
* update to use prod image so that can be validated with build pipeline
* fix typo in comment
* Gangams/enable arc onboarding to ff (#478)
* wip
* updates
* trigger login if the ctx cloud not same as specified cloud
* add missed commit
* Convert PV type dictionary to json for telemetry so it shows up in logs (#480)
* fix 2 windows tasks - 1) Dont log to termination log 2) enable ADX route for containerlogs in windows (for O365) (#482)
* fix ci envvar collection in large pods (#483)
* grwehner/jan agent tasks (#481)
- Windows agent fix to use log filtering settings in config map.
- Error handling for kubelet_utils get_node_capacity in case /metrics/cadvsior endpoint fails.
- Remove env variable for workspace key for windows agent
* updating fbit version and cpu limit (#485)
* reverting to older version (#487)
* Gangams/add fbsettings configurable via configmap (#486)
* wip
* fbit config settings
* add config warn message
* handle one config provided but not other
* fixed pr feedback
* fix copy paste error
* rename config parameter names
* fix typo
* fix fbit crash in helm path
* fix nil check
* Gangams/jan agent release tasks (#484)
* wip
* explicit amd64 affinity for hybrid workloads
* fix space issue
* wip
* revert vscode setting file
* remove per container logs in ci (#488)
* updates for ciprod01112021 release (#489)
* new yaml files (#491)
* Use cloud-specific instrumentation keys (#494)
If APPLICATIONINSIGHTS_AUTH_URL is set/non-empty then the agent will now grab a custom IKey from a URL stored in APPLICATIONINSIGHTS_AUTH_URL
* upgrade apt to latest version (#492)
* upgrade apt to latest version
* fix pr feedback
* Gangams/add support for extension msi for arc k8s cluster (#495)
* wip
* add env var for the arc k8s extension name
* chart update
* extension msi updates
* fix bug
* revert chart and image to prod version
* minor text changes
* image tag to prod
* wip
* wip
* wip
* wip
* final updates
* fix whitespaces
* simplify crd yaml
* Gangams/arm template arc k8s extension (#496)
* arm templates for arc k8s extension
* update to use official extension type name
* update
* add identity property
* add proxyendpointurl parameter
* add default values
* Gangams/aks monitoring via policy (#497)
* enable monitoring through policy
* wip
* handle tags
* wip
* add alias
* wip
* working
* updates
* working
* with deployment name
* doc updates
* doc updates
* fix typo in the docs
* revert to use operatingSystem from osImage for node os telemety (#498)
* Container log v2 schema changes (#499)
* make pod name in mdsd definition as str for consistency. msgp has no type checking, as it has type metadata in it the message itself.
* Add priority class to the daemonsets (#500)
* Add priority class to the daemonsets
Add a priority class for omsagent and have the daemonsets use this
to be sure to schedule the pods.
Daemonset pods are constrained in scheduling to run on specific
nodes. This is done by the daemonset controller. When a node shows
up it will create a pod with a strong affinity to that node. When a
node goes away, it will delete the pod with the node affinity to that
node.
Kubernetes pod scheduling does not know it is a daemonset but it does
know it is tied to a specific node. With default scheduling, it is
possible for the pods to be "frozen out" of a node because the node
already is full. This can happen because "normal" pods may already
exist and are looking for a node to get scheduled on when a node is
added to the cluster. The daemonset controller will only first
create the pod for the node at around the same time. The kubernetes
scheduler is running async from all of this and thus there can be a
race as to who gets scheduled on the node.
The pod priority class (and thus the pod priority) is a way to indicate
that the pod has a higher scheduling priority than a default pod.
By default, all pods are at priority 0. Higher numbers are higher
priority. Setting the priority to something greater than zero will
allow the omsagent daemonsets to win a race against "normal" pods for
scheduled resources on a node - and will also allow for graceful
eviction in the case the node is too full.
Without this, omsagent can be left out of node in clusters that are
very busy, especially in dynamic scaling situations.
I did not test the windows pod as we have no windows clusters.
* CR feedback
* fix node metric issue (#502)
* Bug fixes for Feb release (#504)
* bug fix for mdm metrics with no limits
* fix exception bug
* Gangams/feb 2021 agent bug fix (#505)
* fix npe in getKubeServiceRecords
* use image fields from spec
* fix typo
* cover all cases
* handle scenario only digest specified
* changes for release -ciprod02232021 (#506)
* Gangams/e2e test framework (#503)
* add agent e2e fw and tests
* doc and script updates
* add validation script
* doc updates
* yaml updates
* fix typo
* doc updates
* more doc updates
* add ISTEST for helm chart to use arc conf
* refactor test code
* fix pr feedback
* fix pr feedback
* fix pr feedback
* fix pr feedback
* scrape new kubelet pod count metric name (#508)
* Adding explicit json output to az commands as the script fails if az is configured with Table output #409 (#513)
* Gangams/arc proxy contract and token renewal updates (#511)
* fix issue with crd status updates
* handle renewal token delays
* add proxy contract
* updates for proxy cert for linux
* remove proxycert related changes
* fix whitespace issue
* fix whitespace issue
* remove proxy in arm template
* doc updates for microsoft charts repo release (#512)
* doc updates for microsoft charts repo release
* wip
* Update enable-monitoring.sh (#514)
Line 314 and 343 seems to have trailing spaces for some subscriptions which is exiting the script even for valid scenarios
Co-authored-by: Ganga Mahesh Siddem <gangams@microsoft.com>
* Prometheus scraping from sidecar and OSM changes (#515)
* add liveness timeout for exec (#518)
* chart and other updates (#519)
* Saaror osmdoc (#523)
* Create ReadMe.md
* Update ReadMe.md
* Update ReadMe.md
* Update ReadMe.md
* Update ReadMe.md
* Add files via upload
* Update ReadMe.md
* Update ReadMe.md
* Update ReadMe.md
* Update ReadMe.md
* Update ReadMe.md
* Update ReadMe.md
* telemetry bug fix (#527)
* Fix conflicting logrotate settings (#526)
The node and the omsagent container both have a cron.daily file to rotate certain logs daily. These settings are the same for some files in /var/log (mounted from the node with read/write access), causing the rotation to fail when both try to rotate at the same time. So then the /var/log/*.1 file is written to forever. Since these files are always written to and never rotated, it causes high memory usage on the node after a while.
This fix removes the container logrotate settings for /var/log, which the container does not write to.
* bug fix (#528)
* Gangams/arc ev2 deployment (#522)
* ev2 deployment for arc k8s extension
* fix charts path issue
* rename scripts tar
* add notifications
* fix line endings
* fix line endings
* update with prod repo
* fix file endings
* added liveness and telemetry for telegraf (#517)
* added liveness and telemetry for telegraf
* code transfer
* removed windows liveness probe
* done
* Windows metric fix (#530)
* changes
* about to remove container fix
* moved caching code to existing loop
* removed un-necessary changes
* removed a few more un-necessary changes
* added windows node check
* fixed a bug
* everything works confirmed
* OSM doc update (#533)
* Adding MDM metrics for threshold violation (#531)
* Rashmi/april agent 2021 (#538)
* add Read_from_Head config for all fluentbit tail plugins (#539)
See the commit message of: fluent/fluent-bit@70e33fa
for details explaining the fluentbit change and what Read_from_Head does when set to true.
* fix programdata mount issue on containerd win nodes (#542)
* Update sidecar mem limits (#541)
* David/release 4 22 2021 (#544)
* updating image tag and agent version
* updated liveness probe
* updated release notes again
* fixed date in version file
* 1m, 1m, 1s by default (#543)
* 1m, 1m, 1s by default
* setting default through a different method
* David/aad stage 1 release (#556)
* update to latest omsagent, add eastus2 to mdsd regions
* copied oneagent bits to a CI repository release
* mdsd inmem mode
* yaml for cl scale test
* yaml for cl scale test
* reverting dockerProviderVersion version to 15.0.0
* prepping for release (updated image version, dockerProviderVersion, and release notes
* container log scaletest yamls
* forgot to update image version in chart
* fixing windows tag in dockerfile, changing release notes wording
* missed windows tag in one more place
* forgot to change the windows dockerProviderVersion back
Co-authored-by: Ganga Mahesh Siddem <gangams@microsoft.com>
* Update ReleaseNotes.md (#558)
fix imagetag in the release notes
* Add wait time for telegraf and also force mdm egress to use tls 1.2 (#560)
* Add wait time for telegraf and also force mdm egress to use tls 1.2
* add wait for all telegraf dependencies across all containers (ds & rs)
* remove ssl change so we dont include as part of the other fix until we test with att nodes.
* partially disabled telegraf liveness probe check, we'll still have telemetry but the probe won't fail if telegraf isn't running (#561)
* changes for 05202021 release (#563)
* changes for 05202021 release
* fixed typos
* Rashmi/jedi wireserver (#566)
* Update ReadMe.md (#565)
* Update ReadMe.md
* Update ReadMe.md
Included feedback from OSM team and Fixed
* Gangams/aad stage2 full switch to mdsd (#559)
* full switch to mdsd, upgrade to ruby v1 & omsagent removal
* add odsdirect as fallback option
* cleanup
* cleanup
* move customRegion to stage3
* updates related to containerlog route
* make xml eventschema consistent
* add buffer settings
* address HTTPServerException deprecation in ruby 2.6
* update to official mdsd version
* fix log message issue
* fix pr feedback
* get ridoff unused code from omscommon
* fix pr feedback
* fix pr feedback
* clean up
* clean up
* fix missing conf
* Send perf metrics to MDM from windows daemonset (#568)
* updating json gem to address CVE-2020-10663 (#567)
* updating json gem to address CVE-2020-10663
* updating json gem to address CVE-2020-10663
* update recommended alerts readme (#570)
@dcbrown16 pointed out that this page links to the wrong document in [this issue](https://github.com/microsoft/Docker-Provider/issues/475). The content in the currently linked page is identitical to the page which should be linked, so it's a simple fix.
* trying again to fix the json gem (#571)
* trying again to fix the json gem
* removing installation of newer json gem
* Addressing PR comments for - https://github.com/microsoft/Docker-Provider/pull/568 (#569)
* Mem_Buf_limit is configurable via ConfigMap (#574)
* add log rotation settings for fluentd logs (#577)
* Gangams/release 06112021 (#578)
* updates related to ciprod06112021 release
* minor update
* release note update (#579)
* Make sidecar fluentbit chunk size configurable (#573)
* Fix vulnerabilities (#583)
* test
* test1
* test-2
* test-3
* 3
* 4
* test
* 2
* 3
* 4
* 5
* 6
* rename gem for windows
* fix
* fix
* Windows build optimization (#582)
* fix windows build failure due to msys2 version
* Fix telegraf startup issue when endpoint is unreachable (#587)
* revert fbit tail plugins defaults to std defaults (#586)
* fixed another bug (#593)
* feat: add new metrics to MDM for allocatable % calculation of cpu and memory usage (#584)
* feat: allocatable cpu and memory % metrics for MDM
* maybe
* linux is working
* windwos....
* some more
* comment
* better
* syntax
* ruby
* revert omsagent.yaml
* comments
* pr feedback
* pr feedback
* testing msys2 version update
* better
* update adx sdk for perf issue (#601)
* remove md check
* Gangams/release notes update for hotfix (#596)
* release notes updates
* release notes updates for ciprod06112021-1
* Cherry picking hotfix changes to ci_dev (#605)
* release changes (#607)
* Gangams/aad stage3 msi auth (#585)
* changes related to aad msi auth feature
* use existing envvars
* fix imds token expiry interval
* refactor the windows agent ingestion token code
* code cleanup
* fix build errors
* code clean up
* code clean up
* code clean up
* code clean up
* more refactoring
* fix bug
* fix bug
* add debug logs
* add nil checks
* revert changes
* revert yaml change since this added in aks side
* fix pr feedback
* fix pr feedback
* refine retry code
* update mdsd env as per official build
* cleanup
* update env vars per mdsd
* update with mdsd official build
* skip cert gen & renewal incase of aad msi auth
* add nil check
* cherry windows agent nodeip issue
* fix merge issue
Co-authored-by: rashmichandrashekar <rashmy@microsoft.com>
* Gangams/remove chart version dependency (#589)
* remove chart version dependency
* remove unused code
* fix resource type
* fix
* handle weird cli chars
* update release process
* Gangams/july 2021 release tasks 3 (#613)
* use artifact and pipeline creds for image push
* minor update
* add vuln fix here so that pr can be merged
* remove un-used output plugin (#614)
* fix telegraf telemetry and improve fluentd liveness (#611)
* fix telegraf telemetry and improve fluentd liveness
* address identified vuln with libsystemd0
* fix exported image file extension
* Gangams/july 2021 release tasks 2 (#612)
* tail rs mdsd err logs
* configure mdsd log rotation
* log rotation for mdsd log files
* Fix out_oms.go dependency vulnerabilities (#623)
* revert libsystemd0 update (#616)
* updates for ci-prod release instructions (#619)
* cherry pick changes from ci_prod (#622)
* Support az login for passwords starting with dash ('-') (#626)
Co-authored-by: Vladimir Babichev <vladimir.babichev@finastra.com>
* Gangams/add telemetry fbit settings (#628)
* add telemetry to track fbit settings
* add telemetry to track fbit settings
* check onboarding status (#629)
* Gangams/arc k8s conformance test updates (#617)
* conf test updates
* clean up
* wip
* update with mcr cidev image
* handle log path
* cleanup
* clean up
* wip
* working
* update for mcr image
* minor
* image update
* handle latency of connected cluster resource creation
* update conftest image
* upgrade golang version for windows in pipeline build and locally (#630)
* Updating a link in Readme.md (#632)
The link to the build pipelines now goes directly to our build pipelines (instead of to all github-private pipelines)
* Updating omsagent yaml to have parity with omsagent yaml file in AKS RP (#615)
* Unit test tooling (#625)
Added tooling and examples for unit tests
* run unit tests after a merge too (#634)
* flag stale PRs & issues
* Adding script to collect logs (for troubleshooting) (#636)
* added script for collecting logs
* added windows daemonset and prometheus sidecar, as well as some explanatory prints
* added kubectl describe and kubectl logs output
* changed message to make it more clear some erros are expected
* Sarah/ev2 (#640)
* ev2 artifacts for release pipeline
* update parameters reference
* add artifacts tar file
* changes to rollout and service model
* change agentimage path
* adding agentimage to artifact script
* removing charts from tarball
* change script to use blob storage
* change blob variables
* echo variables
* change blob uri
* use release id for blob prefix
* change to delete blob file
* add check for if blob storage file exists
* fix script errors
* update check for file in storage
* change true check
* comments and change storage account info to pipeline variables
* Changes for windows tar file
* PR changes
* documenting fbit tail plugin configmap settings. (#638)
* documenting fbit tail plugin configmap settings.
* Install unzip package on shell extension (#642)
* Changing installation in ev2 script (#644)
* Adjust release pipeline to use cdpx acr (#647)
* Adjust release pipeline to use cdpx acr
* Adjust release pipeline to use cdpx acr
* Update CDPX ACR path
* Add check for cdpx repo variable
* Sarah/ev2 prod (#649)
* Ev2 changes for prod
* CDPX repo naming change (#652)
* Sarah/ev2 update (#654)
* remove acr name from repo path
* add check to make sure tag does not exist in mcr repo
* change tag syntax for mcr repo check (#655)
* Gangams/optimize win livenessprobe (#653)
* livenessprobe optimization
* optimize windows agent liveness probe
* optimize windows agent liveness probe
* optimize windows agent liveness probe
* optimize windows agent liveness probe
* optimize windows agent liveness probe
* optimize windows agent liveness probe
* optimize windows agent liveness probe
* optimize windows agent liveness probe
* Gangams/addon token adapter image tag to telemetry (#656)
* addon token adapter image tag
* addon token adapter image tag
* Sarah/ev2 helm (#658)
* Use MSI for Arc Release
* Use CIPROD_ACR AME subscription for shell extension
* remove extra line endings
* Sarah/ev2 pipeline (#661)
* testing build artifact dir changes
* add .pipelines directory and omsagent.yaml to build artifacts
* add charts directory to build artifacts (#662)
* Sarah/remove cdpx creds (#664)
* don't use cdpx acr creds from kv
* add e2etest.yaml to build output
* keep cdpx creds for now
* chart updates for rbac api version change (#660)
* chart updates for rbac api version change
* include windows ds for arc
* proxy support (for non-aks) (#665)
* changes related to aad msi auth feature
* use existing envvars
* fix imds token expiry interval
* initial proxy support
* merge?
* cleaning up some files which should've merged differently
* proxy should be working, but most tables don't have any data. About to merge, maybe whatever was wrong is now fixed
* linux AMA proxy works
* about to merge
* proxy support appears to be working, final mdsd build location will still change
* removing some unnecessary changes
* forgot to remove one last change
* redirected mdsd stderr to stdout instead of stdin
* addressing proxy password location comment
Co-authored-by: Ganga Mahesh Siddem <gangams@microsoft.com>
* updates for the release ciprod10082021 and win-ciprod10082021
* updates for the release ciprod10082021 and win-ciprod10082021
* updates for the release ciprod10082021 and win-ciprod10082021
* updates for the release ciprod10082021 and win-ciprod10082021
Co-authored-by: Vishwanath <visnara@microsoft.com>
Co-authored-by: rashmichandrashekar <rashmy@microsoft.com>
Co-authored-by: bragi92 <kadubey@microsoft.com>
Co-authored-by: saaror <31900410+saaror@users.noreply.github.com>
Co-authored-by: Grace Wehner <grace.wehner@microsoft.com>
Co-authored-by: deagraw <deagraw@microsoft.com>
Co-authored-by: David Michelman <daweim0@gmail.com>
Co-authored-by: Michael Sinz <36865706+Michael-Sinz@users.noreply.github.com>
Co-authored-by: Nicolas Yuen <nicolas.yuen@gmail.com>
Co-authored-by: seenu433 <seenu433@gmail.com>
Co-authored-by: Tsubasa Nomura <tsubasa@nomupro.com>
Co-authored-by: Vladimir <vladimir.d.babichev@gmail.com>
Co-authored-by: Vladimir Babichev <vladimir.babichev@finastra.com>
Co-authored-by: sarahpeiffer <46665092+sarahpeiffer@users.noreply.github.com>
* update readme
* make kubecontext optional
* fix minor text issue
* script updates
* script updates
* update solution template paths
* update readme
* script updates
* use local docker provider release shell bundle
* simplify local build process
* update readme
* script update
* doc updates
* comment upgrade command to test in devops pipeline
* run without sudo devops pipeline
* move golang install to end
* set go env at the end
* set go env at the end
* initial pipeline
* update acr registry
* fixed typo
* add version info
* cleanup docker build command
* update ci build agent image
* fix yaml issue
* fix file permission for go
* file permission
* remove sudo and see if that helps
* set rwe permission recursively
* use latest app container image
* move go get to build script
* update build script
* update the build script
* add packages as workaround
* revert makefile changes
* revert changes
* windows agent docker image script placeholder
* use cidev tag for linux agent in the pipeline
* add install pre-req for windows agent
* add install pre-req for windows agent
* fix temp path
* update readme
* add useragent for build agent
* windows build pipeline
* use psscript root instead of pwd
* add restore step since buildstep doesnt have network access
* update to use netcoreapp3.1
* add publish command to restore
* use absolute path for go as workaround
* add publish command back to restore
* add missing envvars in restore
* update env vars
* invoke setx first
* add restore command for go modules
* remove go packages
* dont set gopath variable
* update to version 2.0 container
* update readme
* use 3.0 version
* update 4.0 windows build agent
* update to use version 5.0 windows container image
* use --no-restore option
* use absolute path for go
* set explicit envvars
* remove --no-restore option
* final windows agent
* update to use latest windows build agent machine
* simplify local build for windows
* update gitignore file
* update the readme
* add newline for missing ones
* update readme with the instructions to use the code on wsl for windows agent development
* minor cleanup
* final readme updates
* fix pr feedback
* fix label
* added maintainer and label for windows image
* add docker.version file
* move installer files from pal directory
* update configure file
* wip
* clean up makefile and installer datafiles
* fix ai import casing
* re add data files
* update makefile
* update make file
* remove super project git references
* remove legacy host agent code
* merge windows code
* add docs, troubleshoot and alerting
* move windows code to source dir
* refactor the scripts
* remove unnecessary files
* fix path certificategenerator.zip
* bring windows code back
* get latest changes from ci_feature
* remove redundant files
* fix liveness probe path issue
* rename the dir names
* clean up
* clean up
* omi dependencies
* remove redundant files
* update release notes
* merge latest changes for 05222020
* removed weird whitespaces
* update readme with windows version
* update readme with windows agent build instructions
* reorganize scripts
* minor update
* update readme
* re-oragnize the code structure and add build script for windows agent
* update readme
* updates to readme and clean up
* fix build errors
* fix build errors
* fix build errors
* update readme
* update main and setup scripts
* fix path issue in windows docker file
* readme updates
* windows agent path issue
* resolve ai package name conflict
* rename things
* rename file names to have uniform casing
* get ridoff glide files
* add version file
* merge latest chanegs in ci_feature branch
* do go get as part of the build
* doc updates
* read me update
* update docker file
* reorganize the windows code
* reorganize windows code
* update gitignore to not include files under windows
* clean up files
* update readme file
* update to use version info for windows agent
* merge Makefile.common into Makefile
* fix build issues
* fix build error
* add sudo for go commands
* remove go get from makefile
* update to use go1.14.1
* fix bug in windows Makefile script
* fix build error
* remove weird char in makefile.ps1
* readme update
* fix windows build issue
* read me update
* version update to synch with ciprod05262020 release
* add omsagent-ai-res-id yaml
* build with -buildmode=c-shared for out_oms.so for windows agent
* final yaml updates
* use version info for linux go so file
* fix build error
* readme updates
* update readme
* add shell script for installing for linux agent pre-requisites
* take latest release notes
* readme update
* disable monitoring addon script
* pr feedback
* pr feedback
* update disable addon script
* clean up code
* clean up commented code
* pr feedback
* wip
* wip
* refactor scripts to managed
* wip
* cleanup scripts
* update for aks
* add more validation
* Wip
* update powershell script
* add disable monitoring powershell script
* fix bugs with disable script
* fix bugs in ps scripts
* update readme
* re-arrange source code
* re-arrange test code
* update readme
* update path
* move go code under src dir
* update build dependencies to work on wsl
* update readme with code of conduct
* final readme updates
Co-authored-by: root <root@gangams-ci-1.localdomain>
Janvi Jatakia
Alexander Kaplan
Amol Agrawal
Ganga Mahesh Siddem
Vishwanath
r-dilip
Jeff Coffler