Граф коммитов

121 Коммитов

Автор SHA1 Сообщение Дата
Ganga Mahesh Siddem 8a1ef67cc3
Gangams/multi tenancy add advanced_mode_enabled option in the configmap (#1328)
* Contianer insights 3.1.24 release charts update and release notes

* add azmon multi-tenancy advanced mode option

* revert release related updates

* revert release related changes

* improve comments

* improve logging

* get multitenancy config only in msi mode

* comment multi-tenancy yaml

* cleanup unnecessary logging

---------

Co-authored-by: longwan <longwan@microsoft.com>
2024-10-07 09:16:06 -07:00
Ganga Mahesh Siddem c1fa9a6870
increase retry limit (#1327) 2024-10-02 17:16:45 +00:00
Long Wan 4f02244f35
fix windows ama toggles default bug (#1320) 2024-09-21 00:15:31 +00:00
Ganga Mahesh Siddem bcdd3581dc
Gangams/logs multi tenancy with service (#1309)
* dcr and cm updates

* fix whitespace

* arm templates for multi-tenancy logs onboarding

* initial implementation

* implement refresh logic

* fix bug

* ignore vuln

* improvements

* handle routing multiple DCRs for a namespace

* refactor code

* cleanup code

* cleanup code

* add telemetry

* bug fixes

* code cleanup

* refactor code

* refactor code

* refactor code

* refactor code

* clean up

* clean up

* clean up

* fix bug

* add dce as dependency for dcr

* remove cm setting until we get the public preview

* add telemetry

* fix bug

* fix config size bug

* fix bug

* fix minor logging

* fix multi-tenancy bug in windows

* fix build error

* fix bug

* fix bug

* minor change

* minor change

* fix bugs

* fix build error

* additional logs

* additional logs

* additional logs

* more logging

* fix bugs

* fix bug

* better naming

* clean up

* fix pr feedback

* fix pr feedback

* minor

* refactor code

* refactor code

* fix bug

* fix bug

* refactor code

* clean up

* use latest api version for dcr,dce and dcra

* add ampls support

* add high log scale mode check for multi-tenancy

* fix minor comment

* fix windows bug

* azmon multi-tenancy conf files

* fix bug

* initial implementation

* fix bug

* add config customizer support

* use defaults

* add support for ama logs service

* yaml updates for ama-logs-multitenancy service and deployment

* minor changes

* fix build error

* update trivy ignore

* fix crash

* bug fixes

* add logging

* set multi-tenancy settings only when enabled

* clean up

* refactor conf

* refactor conf

* conf and yaml updates

* throttle window 5min

* wip

* tenant namespace level config

* revert conf customizer changes

* revert conf customizer changes

* clean up

* clean up

* add service side settings

* clean up

* refactor code

* fix exception

* fix bug

* fix bugs

* add support for default namespace level settings

* refactor code

* clean up code

* handle geneva backward compat

* fix the bug

* fix bug

* fix bug

* add multi-tenancy support for windows

* grafana dashboards

* dashboard updates

* update grafana dashboards

* bump default limits for ama-logs-multitenancy

* grafana dashboards

* grafana dashboard updates

* fix pr feedback

* fix pr feedback

* fix pr feedback

* fix pr feedback

* refactor code

* fix pr feedback

* refactor code

* fix build error

* update trivy ignore

* fix bug in livenessprobe

* fix bug

* add disable throttle config option

* fix bug

* fix bug

* cleanup configmap

* yaml updates

* improve comments in cm

* improve comments in cm

* improve minor comments

* minor update to trigger build
2024-09-20 21:03:50 +00:00
Long Wan 510da26693
Fix resource optimization bug (#1317)
* fix resource optimization bug

* address commments
2024-09-19 14:54:16 -07:00
Long Wan 2144018410
enable telegraf livenessprobe for linux (#1299) 2024-07-27 15:47:31 -07:00
Amol Agrawal 46645cc853
remove adal go package and disable ADX log route (#1294)
* remove adal go package and disable ADX log route

---------

Co-authored-by: Amol Agrawal <amagraw@microsoft.com>
2024-07-16 09:17:41 -07:00
Amol Agrawal 1980786938
enable syslog collection for cosmic (#1282)
* enable syslog collection for cosmic

---------

Co-authored-by: Amol Agrawal <amagraw@microsoft.com>
2024-06-20 17:41:47 +00:00
Ganga Mahesh Siddem 8e3a8a8cf5
Bump github.com/Azure/azure-sdk-for-go/sdk/azidentity (#1281)
Bumps [github.com/Azure/azure-sdk-for-go/sdk/azidentity](https://github.com/Azure/azure-sdk-for-go) from 1.5.1 to 1.6.0.
- [Release notes](https://github.com/Azure/azure-sdk-for-go/releases)
- [Changelog](https://github.com/Azure/azure-sdk-for-go/blob/main/documentation/release.md)
- [Commits](https://github.com/Azure/azure-sdk-for-go/compare/sdk/internal/v1.5.1...sdk/azcore/v1.6.0)

---
updated-dependencies:
- dependency-name: github.com/Azure/azure-sdk-for-go/sdk/azidentity
  dependency-type: indirect
...

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2024-06-19 17:05:52 -07:00
Janvi Jatakia (from Dev Box) 10c23ce88b Update to use red color for error messages 2024-05-23 17:16:55 -07:00
Amol Agrawal d539909eec
disable custom metrics (#1260)
* disable custom metrics by default

---------

Co-authored-by: Amol Agrawal <amagraw@microsoft.com>
2024-05-23 16:47:11 -07:00
Ganga Mahesh Siddem 5ced912fa4
sync yamls, agent log level to info and remove redundant test http server (#1256)
* updates

* update trivy ignore

* update the addon-token-adapter image
2024-05-20 11:32:04 -07:00
Ganga Mahesh Siddem 84fb709105
Gangams/logs 50k eps per node (#1235)
* mdsd version 50k changes

* amacore agent integration

* update liveness probe

* handle non-existent file

* refactor code

* fix bugs in mdsd install

* add poll to check amaca port up and running

* fix bug

* configure amaca configport

* try released mdsd version 1.30.3

* fix bug in logs and events profile

* test latest version of mdsd in GIG mode for both arm and x64

* try with build 50k eps changes

* update templates for high log scale mode

* remove libc.so copying

* revert logrotate conf for amaca log

* update mdsd version which has crash fix

* add proxy support for amacore agent

* update mdsd build with amaca gig la changes

* update mdsd build with gig la fixes

* update windows ama build

* mdsd version with 25k buffer size in mdsd

* update mdsd build

* add telemetry and configmap option

* fix bugs

* windows ama build with resource id bug fix

* update mdsd version with qos fixes

* update to use working templates

* add frequency to control amaca log

* mdsd build with qos updates

* trivy ignore update

* log amaca agent version

* improve comments

* add default fluent-bit config for high log scale

* add threding on tail plugin when high log scale enabled

* fix bugs

* fix bug

* fix bugs

* some improvements

* improve comments

* improve code

* update trivyignore

* fix bug

* update trivyignore

* pick GIGLA stream from config when highlogscale enabled

* fix bug

* template updates for high log scale mode

* fix bug

* clean up

* set envvar for ishighlogscale

* set envvar for ishighlogscale

* fix bug

* add log message to troubleshoot duplicate logs

* add log message to troubleshoot duplicate logs

* handle ama bug until fixed

* add storage total limit size

* rename for better reading

* fix pr feedback

* fix pr feedback

* fix pr feedback

* mdsd version update

* fix proxy bug

* fix proxy bug

* update trivy ignore

* clean up the code

* refactor code

* increase storage limit size to 2GB

* increase storage limit size to 10GB

* official mdsd and windows ama versions

* code cleanup

* code cleanup

* mdsd version annotation update

* fix pr feedback

* fix pr feedback

* fix pr feedback

* fix pr feedback
2024-05-09 16:48:56 -07:00
Amol Agrawal 48ee32d5dc
streamline input plugin code. (#1238)
* streamline input plugin code

---------

Co-authored-by: Amol Agrawal <amagraw@microsoft.com>
2024-05-02 08:39:05 -07:00
Janvi Jatakia 3646eaf05b
Geneva -send windows container inventory and perf with RS (#1233)
* Update the geneva feature flag for RS

---------

Co-authored-by: Janvi Jatakia (from Dev Box) <jajataki@microsoft.com>
2024-04-26 17:19:52 -07:00
Amol Agrawal cba115c7bf
update dcr optimization error messages (#1228)
* update dcr optimization error messages

* add additional check for geneva

* redirect dcr parser stderr and stdout to traces file

---------

Co-authored-by: Amol Agrawal <amagraw@microsoft.com>
2024-04-19 18:18:06 +00:00
Ganga Mahesh Siddem 860d1e816d
fix bugs (#1230)
* fix bugs

* fix comment
2024-04-18 15:47:29 -07:00
Amol Agrawal 05e094df24
reduce podsChunkSizeMin (#1225) (#1226)
Co-authored-by: Amol Agrawal <amagraw@microsoft.com>
2024-04-12 00:40:05 +00:00
Amol Agrawal 7a14cbec50
collect logs from pods from excluded system ns (#1138) (#1146)
* collect logs from pods from excluded system ns
2024-03-21 17:02:35 -07:00
Long Wan 33b1b6a957
Containerlogv2 Metadata and annotations based filtering (#1148)
* add fluent-bit kubernetes filter (#1115)

* add fluent-bit kubernetes filter

---------

Co-authored-by: Amol Agrawal <amagraw@microsoft.com>

* add metadata feilds including labels, annotations, uid and image (#1120)

* add metadata feilds including labels, annotations, uid and image

* set the include_fields if customer set it otherwise go with default if enabled

* fix minor bugs

* change placeholder

* add exclude specific kubernetes config

* update tls verify for linux

* make log filtering opt in explicit (#1140)

Co-authored-by: Amol Agrawal <amagraw@microsoft.com>

* Longw/metadata containerlogv2 kubernetes (#1139)

* add metadata feilds including labels, annotations, uid and image

* rename to podUid and add nil check

* set the include_fields if customer set it otherwise go with default if enabled

* add workload to testing clusters

* containerlogv2 metadata backend change with feature flag on

* adjust the marshal and log the output

* address comments add check and scenario for empty list

* remove ADX support

* go fmt for better format

* fix space in tomlparser.rb

* update trivy ignore

* fix  Kube_Tag_Prefix for windows

* update trivy

* add ttl for metadata cache

* update containerlogv2 test workloads

* change typo and update tests

* Longw/metadata containerlogv2 address feedback (#1166)

* add argument for sendMetric

* update SendMetric

* update SendMetric

* Longw/metadata containerlogv2 address feedback2 (#1186)

* extend to support more configs on metadata

* add warning message if feilds not match

* add plugin for geneva path

* add kubernetesMetadataCollection for geneva path

* Longw/metadata containerlogv2 address feedback2 (#1188)

* remove send metric

* update trivy

* Longw/metadata containerlogv2 address feedback3 (#1195)

* update doc for containerlogv2 linux

* add kube_meta_cache_ttl to agent settings

* update logic to adjust colonLocation as the end of the list if it is not found

* address comments and feedback

* update conf file for geneva

* adjust geneva and add new metrics

* address feedback for geneva path

* add telemetry in go

* address comments for geneva conf filter config

---------

Co-authored-by: Amol Agrawal <pfrcks@gmail.com>
Co-authored-by: Amol Agrawal <amagraw@microsoft.com>
2024-03-19 09:09:07 -07:00
Ganga Mahesh Siddem 83946e7e48
start telegraf only when fluent tcp listener up and running (#1205)
* start telegraf only when fluent tcp listener up and running

* remove duplicate plugin

* update trivy ignore

* consistent with linux
2024-03-18 10:06:02 -07:00
Janvi Jatakia afecf342d6
Update windows_fluent_bit default to disable (#1201)
* Update the default to disable

* Update to use rs for no value of env as well

---------

Co-authored-by: Janvi Jatakia (from Dev Box) <jajataki@microsoft.com>
2024-03-12 08:53:27 -07:00
Janvi Jatakia ad98269a64
Update the code to take toggle for env vars in windows (#1196)
Co-authored-by: Janvi Jatakia (from Dev Box) <jajataki@microsoft.com>
2024-03-06 19:50:45 +00:00
Janvi Jatakia 063d35cae0
Integrating fluent-bit input plugins for Linux/Windows (#1144)
* Implement input plugin setup + perf and containerinventory plugins (#1112)

* migrate perf and containterinventory plugins

---------

Co-authored-by: Amol Agrawal <amagraw@microsoft.com>

* add var to livenessprobe (#1116)

Co-authored-by: Amol Agrawal <amagraw@microsoft.com>

* Add perf and container inventory plugins for Windows and integrating them to named pipe (#1113)

Add perf and container inventory plugins for Windows and integrating them to named pipe (#1113)

* Resolving comments

* Solving a bug

* Using the new input plugins only for windows msi mode

* Remove the duplicate flow of logs

* Not update the config files when legacy and resource optimization is on

* Not updating the config map for geneva as well

* Solving timestamp bug and add windows_rs.conf files

* Var name change bug

* Use the stat of a file instead of reading from file

* address PR comments - I

* update fluent-bit to 2.1.10

* update trivyignore

* fix bug in confg parsing

* add shared library

* revert fluent-bit upgrade in windows

* Adding a configmap option for disabling windows fluent-bit

* Adding a configmap option for disabling windows fluent-bit

* Solving a bug

* Resolving comments

* Solving RS bug

* Solving a bug

* Resolving comments

* remove linux option from cm

* Update trivyignore

* Sign the containerinventory.so and perf.so files

* Signing windows binary files

* Update the go mod tidy for the ci_prod changes

* Update go.mod to 1.21

* bug

* Update fluent-bit conf

* Update the fluent-bit conf file

* fluentd starting two times

---------

Co-authored-by: Amol Agrawal <pfrcks@gmail.com>
Co-authored-by: Amol Agrawal <amagraw@microsoft.com>
Co-authored-by: Janvi Jatakia (from Dev Box) <jajataki@microsoft.com>
2024-03-05 09:57:14 -08:00
Ganga Mahesh Siddem bb3a78eb38
fix configmap parse error (#1191) 2024-03-04 10:35:39 -08:00
Ganga Mahesh Siddem 1ac199319c
Gangams/expose disable telemetry config option (#1189)
* config option for telemetry disablement

* Update .trivyignore

* changes related to disable telemetry config option

* fix pr feedback

* clean up comment

* refactor code
2024-03-02 03:57:21 +00:00
Amol Agrawal a50bc7298f
update multiline parser config logic (#1180)
* update parser setup

---------

Co-authored-by: Amol Agrawal <amagraw@microsoft.com>
2024-02-28 10:42:13 -08:00
Amol Agrawal 61e51b4b36
update v2 schema configmap (#1175)
Co-authored-by: Amol Agrawal <amagraw@microsoft.com>
2024-02-28 10:10:00 -08:00
Ganga Mahesh Siddem 0c4daabdfc
fix telegraf config related fixes (#1176)
* fix telegraf config related fixes

* remove unused bearer_token from config

* fix pr feedback
2024-02-28 03:00:50 +00:00
Amol Agrawal d388be9d9b
explicity set LOGS_AND_EVENTS_ONLY (#1177)
* explicity set LOGS_AND_EVENTS_ONLY

* make path explicit

---------

Co-authored-by: Amol Agrawal <amagraw@microsoft.com>
2024-02-27 09:01:52 -08:00
Amol Agrawal 5f98930640
make multiline filter first in pipeline (#1151)
Co-authored-by: Amol Agrawal <amagraw@microsoft.com>
2024-02-23 10:19:06 -08:00
Amol Agrawal 4c93140353
fix bugs for geneva and legacy mode for toggle components (#1174)
* fix bugs in toggle components code flow

---------

Co-authored-by: Amol Agrawal <amagraw@microsoft.com>
2024-02-21 21:19:53 +00:00
Amol Agrawal e6a19e5b02
Disable component based on logs profile (#1141) (#1145)
* disables fluentd and telegraf in linux ds if logs or events only streams enabled

---------

Co-authored-by: Amol Agrawal <amagraw@microsoft.com>

* address PR comments

* added telemetry

---------

Co-authored-by: Amol Agrawal <amagraw@microsoft.com>
2024-02-20 16:01:24 -08:00
Ganga Mahesh Siddem 71bac29ba8
Gangams/telegraf update to 1.28.5 (#1159)
* telegraf update from 1.27.3 to 1.28.5

* telegraf update from 1.27.3 to 1.28.5

* comment bearer_token in telegraf-prom-side-car.conf since its not required

* telegraf update for windows

* update from response_timeout to timeout in conf files

* add timebound token changes

* fix comment

* telegraf errors to telegraf log file

* test yaml

* since 1.28.5 doesnt work for windows trying with 1.29.0

* revert telegraf changes for windows

* enable telegraf debug logs

* revert debug log enablement

* resolve merge conflicts

* update code comments

* revert unneeded change
2024-02-12 12:14:54 -08:00
Ganga Mahesh Siddem 993b4c2272
Gangams/fix gh issue 3999 (#1168)
* specify mountPropagation HostToContainer

* remove docker sock dependency

* update trivy ignore
2024-02-12 09:23:13 -08:00
Amol Agrawal ef7f4a0782
make multiline languages configurable (#1149)
* make multiline languages configurable

---------

Co-authored-by: Amol Agrawal <amagraw@microsoft.com>
2024-01-11 11:20:12 -08:00
Ganga Mahesh Siddem f1dc056d22
Gangams/dotnet core version update (#1110)
* dotnet version update from 3.1 to 7.0

* dotnet version update from 3.1 to 7.0

* fix pr feedback
2023-11-03 12:13:49 -07:00
Long Wan f8ea86ccb5
Longw/esrp signing and dynamic scripts refactoring (#1087)
Binaries and images signing for container insights
2023-10-19 16:01:14 -07:00
Amol Agrawal 49c2750679
syslog fix mounting bug (#1095)
* fix multiple mount bug

* dont start syslog port in prom sidecar
2023-10-09 13:36:56 -07:00
Janvi Jatakia 0a4faf8a53
Add tailing of the MonAgentHost logs (#1094)
* Add tailing of the MonAgentHost logs

* Resolving comments

---------

Co-authored-by: Janvi Jatakia (from Dev Box) <jajataki@microsoft.com>
2023-10-06 14:47:23 -07:00
Amol Agrawal f4caa8ea2b
use omfwd for syslog (#1093)
* use omfwd

* update queue.size and use SYSLOG_HOST_PORT value

* use tdnf for mdsd

---------

Co-authored-by: Amol Agrawal <amagraw@microsoft.com>
2023-10-06 12:31:01 -07:00
Janvi Jatakia 04df7e307a
Windows AMA <> Container Log/LogV2 integration (#1071)
* Updating main.ps1 and setup.ps1

* Updating the extension.go code and adding Linux and WIndows specific codes

* Updating oms, telemetry and utils_windows files

* Update utils_linux.go

* Remove unnecessary oms.go changes

* Update extension_linux to return nil instead of ""

* Add aksRegion and aksResourceId to custom vars

* Update to see what is the response from extension

* Changing taggeddata version to string

* Add useFromCache functionality

* Updating containerlogv2 shema from dcr

* Updated linux unimplemented function

* Add remaining imports

* Update useFromCache for containerLogV2

* livenessprobe changes

* Resolving comments

* Updated the version number using properties of file

* Update the function name in Linux files as well

* Update main.ps1 for cert generation in case of Geneva mode

* Update main.ps1 error and add ama-logs

* Update the windows ama msi mode condition

---------

Co-authored-by: Janvi Jatakia (from Dev Box) <jajataki@microsoft.com>
2023-10-05 19:20:40 -07:00
Ganga Mahesh Siddem 8d4380dd27
add separate flags for windows and linux in configmap for geneva logs integration (#1064) 2023-09-13 16:19:08 -07:00
Ganga Mahesh Siddem 90f2f49644
Gangams/make infra geneva config optional in multi tenancy (#1050)
* make geneva config optional

* include namespace for geneva telemetry service pod

* remove unused prom scrape annotations

* remove unused prom scrape annotations

* fix bug

* fix pr feedback

* fix pr feedback
2023-08-15 09:18:04 -07:00
Amol Agrawal f476b3a794
set mdsd limits (#1027)
* set mdsd limit using container memory limit

---------

Co-authored-by: Amol Agrawal <amagraw@microsoft.com>
2023-08-08 14:43:47 -07:00
Ganga Mahesh Siddem d5f9bb7c5b
Gangams/expose mdsd config for high log customers (#1037)
* add mdsd settings related to high log scale

* prom config

* prom config

* grafana json update

* grafana json update

* add cluster filter to prom queries

* Update ama-metrics-prometheus-config.yaml

* Grafana for external

* add compression level config

* remove unused prom annotations

* comment mdsd config

* comment mdsd config

* comment mdsd config

* update trivy ignore

* fix bug

* update trivy ignore

* fix pr feedback

* add comment to configmap
2023-07-27 15:23:28 -07:00
Ganga Mahesh Siddem ca2b678355
Gangams/disable threaded by default and make optional (#1028)
* disable threading and making it default

* disable threading and making it default
2023-07-20 10:41:02 -07:00
Amol Agrawal 7a21c7b500
multiline add java and python stacktrace (#1012)
* add java and python to fluent-bit multiline
2023-07-06 14:39:31 -07:00
Janvi Jatakia cb17f32e34
Muting telegraf for Windows if monitor_kubernetes_pods = false or N/A (#985)
* removing trivyignore

* Adding the environment var for monitorKubernetesPods to start telegraf only when true

* Update the main.ps1

---------

Co-authored-by: Janvi Jatakia <jajataki@microsoft.com>
2023-06-06 11:40:58 -07:00
dependabot[bot] d62be1af2f
Bump BouncyCastle in /build/windows/installer/certificategenerator (#979)
Bumps BouncyCastle from 1.8.6.1 to 1.8.9.

---
updated-dependencies:
- dependency-name: BouncyCastle
  dependency-type: direct:production
...

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Co-authored-by: Ganga Mahesh Siddem <gangams@microsoft.com>
2023-06-01 21:29:54 -07:00