* add L1VH IB support on CNI
* fix IB issues
* fix UT errors
* fix linter issues
* add win 2025 support for cni image build
* add and comments
* fix a logic bug
* disable endpoint creation and deletion if it's IB NIC
* fix a linter issue
* add UTs
* add UTs for powershell
* enhance Test_getInterfaceInfoKey test case
* remove windows 2025 build from pipeline
* fix some issues
* add an UT to test pnpID
* fix an issue
* fix an ut
* add double quotes
* unblock a brunch of issues
* remove unnecessary codes
* upgradelatest upstream cnii build
* fix a log
* add windows build on pipeline temporarily
* remove backendNIC check for findMasterInterface
* add ut to confirm IB does not create endpoint
* fix linter issue that use %q
* format network.go
* add more uts to cover powershell commands
* remove windows2025 pipeline build
* enhance logs
* fix cniResult format
* add getPnpidstate func
* fix the issue for infraNIC routes
* fix the issue for infraNIC routes
* fix gateway ip address
* add get-pnpdevice UT
* add accelnetNIC support for L1VH
* enhance logic for accelnet nic netowrk flag
* enhance network windows uts
* fix bitmask operator
* use another PR for accelnet PR
* gofumpt files
* fix comments for functional codes
* add uts
* add more uts
* fix uts
* fix functional codes comments
* Update cni/network/network.go
Co-authored-by: tamilmani1989 <tamanoha@microsoft.com>
Signed-off-by: Paul Yu <129891899+paulyufan2@users.noreply.github.com>
* fix latest comments
* fix an UT
* fix invoker_cns_test.go
* fix ut bugs
* fix ut with SkipDefaultRoutes
* add combination ut
* add combination ut
* add ncGateway address to ut
* fix an ut bug
* fix ut bug
* add unhappy test cases
* add endpoint add and deletion cases
* push mock network creation hns api test cases
* remove network creation hns call
* add uts to mock hns network and endpoint calls
* fix ut linter issues
* add infraNIC only invoker test case
* add unhappy path test case
* remove infraNIC only case
* remove unhappy test case
* re-archetect cni ib codes and test
* remove unnecessary logs
* save endpoint state
* save endpoint object for IB
* fix linter issue
* fix a brunch of linter issues
* fix linter issues
* fix linter issue
* fix ut for returned error msg
* temporary add manifest build for CNS/CNI to pipeline
* feedback fix
* fix linter issue
* add ut to get networkName and networkID
* remove Ankit's PR to build cns image
* revert Ankit's changes back
* remove win2025 build from pipeline
* log error for invalid mac address
* revert convertInterfaceInfoToCniResult impl
* fix feedback
* add crd changes to test
* add win2025 yaml to build image
* pass containerID to cns
* revert changes back for review
* revert changes back for review
* gofumpt endpoint.go
* remove comment
* add latest comments
* Update network/endpoint_windows.go
Co-authored-by: tamilmani1989 <tamanoha@microsoft.com>
Signed-off-by: Paul Yu <129891899+paulyufan2@users.noreply.github.com>
* fix a linter issue
* add error check
* add error check
* gofumpt endpoint windows test file
---------
Signed-off-by: Paul Yu <129891899+paulyufan2@users.noreply.github.com>
Co-authored-by: tamilmani1989 <tamanoha@microsoft.com>
* feat: add UseMTLS config
* feat: add mTLS auth for CNS
* test: add testdata for mTLS tests
* chore: add logs on TLS config retrieval
* lint: in tests
* refactor: use CNS logger, not ACN logger
* refactor: add guards to mtlsRootCAsFromCertificate and unit tests
* lint: fix lint errors
* test: include HTTP listener tests for when TLS/mTLS is enabled
* chore: add log for stopping the TLS listener
* test: add test helper to create certificates for testing instead of using hardcoded pem file
* test: assert non-TLS service has no TLSSettings
* test: refactor TestMtlsRootCAsFromCertificate to table-based tests
* refactor: pull listener addresses from listener and remove redundant struct field for tls address
* start two cns servers
* fix linter issues
* fix linter issue
* enhance echo server log
* fix Quang's comments
* gofumpt cns service.go
* fix defaultAPIServerAddress
* run servers asynchoronsly
* add server_test.go
* fix UT test
* enhance log
* fix log issue
* fix an issue
* add a check for defaultAPIServerURL
* fix GetOption func
* enhance comments
* fix defaultAPI URL to 127.0.0.1
* fix restapi method in echo server
* remove unnecessary log info
* fix comment1
* fix comments
* add context to control echo server
* fix TM's comments
* do not enable local server if user specifies cnsurl by -c option
* add -p port check when getting nodeURL
* fix comments and use -p select from customer to start local server
* fix a case when customer does not provide -p option
* fix the issue if customer also does not provide -p and -c option
* add UTs to launch server with different combinations
* fix linter issue for UTs
* fix UT linter issues
* UT needs to stop service
* remove duplicated test case
* comments fix
* fix linter issue
* fix Tim's comments
* fix comments
* fix linter issue
* handle type assertion
* Adding cnsURL and cnsPort options to test server
The startService function is a mistake that has been perpetuated through
time and needs to be removed. As a consequence of its existence, it
requires mixing concerns in all tests and stands in the way of
refactoring.
The addition of two type assertion checks caused this function to break,
because the corresponding options were not set in the Service's
configuration. This is a problem in itself--there's no reason the
options should be pulled out of a map[string]interface{}. Given this, we
can just set these two options in startService, committing sins to
offset the ones in service.(*HTTPRestService).Service.Options.
---------
Signed-off-by: Paul Yu <129891899+paulyufan2@users.noreply.github.com>
Co-authored-by: Tim Raymond <traymond@microsoft.com>
* feat: 🌈 StatelessCNI: Adding getEndpoint and UpdateEndpoint API to CNS (#2102)
* Adding getEndpoint and UpdateEndpoint API to CNS with the respective clients in support of stateless CNI.
* Updating the unit tests and address the comments.
* Addressing the comments.
* Addressing the coments regarding CNS support for Stateless CNI
* Adddressing the PR comments
* 🌈 feat: adding flags for stateless cni (#2103)
feat: stateless cni
* 🌈 feat: StatelessCNI: Applying stateless CNI mode changes in network package. (#2197)
* Apllying stateless CNI mode in network package.
* Addresing the commetns.
* feat: create stateless cni binary for swift (#2275)
* enabling CNS telemetry
* Master rebase changes
* CNI Telemetry enabled on CNS
* Stateless CNI changes.
* making change to CNSendpointStorePath
* Updating makefile to avoid creating statless CNI release.
---------
Co-authored-by: Vipul Singh <vipul21sept@gmail.com>
* feat: added logger package and replaced old log initialization for the new one
* feat: changed all log lines to new zap logger
* fix: typo
* Update azure-ipam/logger/logger.go
Co-authored-by: Timothy J. Raymond <timraymond@users.noreply.github.com>
* fix: adding constants to describe logger rotations constraints
* Renamed logger New method
* Replaced logAndSend method by log 1st and send then so we can use zap fields
* minor fixes
* added logger init for tests
* replaced Any by Error
* gci ipam_test
* fixed govet errors
* moved component to a zap field
* fixed linit issues
* added log mock
* fix: gci
* fix: added context for logger teardown
* Update cni/log/logger.go
Co-authored-by: Evan Baker <rbtr@users.noreply.github.com>
* moved logger init mock function
* fix: lint findings
---------
Co-authored-by: Esteban Capillo <estebancams@microsoft.com>
Co-authored-by: Timothy J. Raymond <timraymond@users.noreply.github.com>
Co-authored-by: Evan Baker <rbtr@users.noreply.github.com>
* wip with StrictlyHasSetPolicies approach
* better approaching of getting all set policies
* wip for rigorous win dp UTs
* marshal setpolicies in hns mock and dont short circuit in UTs
* policy stuff and update test cases
* marshal ACLs in hns mock
* more UTs and minor refinements
* option to apply dp or not
* address cmp.Equal and t.Helper comments
* dpEvent returns error and better defined concurrency
* remove unnecessary logic in concurrent test code
* approach #3 emulating cyclonus
* namespace method for podmetadata
* refactor Action structure and TestCase wait group behavior
* hnsactions and renaming a file
* refactor to Serial and ThreadedTestCase structs, and move files to dp pkg
* hns latency hard coded to be the same for all threaded test cases
* fix build error after rebasing
* export fake hns network id
* address comments on multierr and terminology
* add comment about pod metadata in controller
* pod update and delete actions
* move ApplyDPAction to top
* namespace actions and rename some fields of UpdatePod
* adding code comments
* reconcile action
* fix bug in key-val ipsets
* implement all previous test cases
* fix incorrect error wrapping in dataplane.go
* multi-job tests are working. updated terminology from routine to job
* MultiErrManager instead of dependency for multierr
* return to the channel approach for multierr, now using FailNow instead of asserting on channel length
* fix some lints
* fix more lints
* rebase
* linting
* rebase
* missing if condition for releaseIPConfig
* update azure-cns.yaml and add UTs
* rebase
* update program iptables changes
* linting
* fix broken tests
* fix podinfoprovider returns error when key is not found
* log when no endpoint state exist when reconcilling
* not remove endpoint state file on failure to read in restserver.restoreState()
* addressed comments
* update acn tag
* go get on acn
* addressed comments
Co-authored-by: Evan Baker <rbtr@users.noreply.github.com>
* Remove azure-vne-telemetry for windows multitenancy and telemetry service for windows multitenancy will be started from cns.
* start telemetry service from cns
* lint and log fix
* minor change
* addressed comment
* [NPM] Expanding HNS fake usage with internal state
* adding some test cases for windows DP
* adding some test cases for windows DP
* Correcting some issues with windows DP
* Splitting each modify call into two, one for 1st level sets and another for nested sets
* Fixing a build issue in linux
* Enhancing windows ipset tests
* Adding ipset mgr windows tests
* fixing a build issue
* Adding in ACL verify code
* addressing some lint issues
* addressing some lint issues
* removing apply ipset in generic ipsetMgr tests
* fixing a build issue with windows
* fixing windows build issue
The io/ioutil package has been deprecated as of Go 1.16, see
https://golang.org/doc/go1.16#ioutil. This commit replaces the existing
io/ioutil functions with their new definitions in io and os packages.
Signed-off-by: Eng Zer Jun <engzerjun@gmail.com>
* use ipset save to update members and update error handling logic for ipsets to skip previously run lines. Also update some logging for iptables chain management
* remove unused code
* add comment block describing high level ipset restore logic
* fix bug in piping to grep. need to add pipe errors for fexec UTs so that we dont revert on dataplane UTs
* grep for npm sets working for ipsets save, but this breaks DP UTs
* VerifyCalls method for mock ioshim
* add ability to unit test piped commands
* update logging
* UTs for piping a command to grep
* grep for npm sets in ipset save, verify number of calls in UTs, and update ApplyIPSets test calls for dataplane UTs
* update comments based on PR suggestions
* addressing comments
* remove out-of-scope policy changes for this PR
* rename restore file creator files
* FIXME: setting v2 controllers toggle to true to create an image in pipeline
* Revert "FIXME: setting v2 controllers toggle to true to create an image in pipeline"
This reverts commit 31148c3034.
* wrap errors
* [NPM] Adding prefixes to IPSets in dataplane
* Correcting a linting issue
* Using the correct case for metadata
* Adding IOShim for both windows and linux
* splitting ioshim for each os
* correcting a import error
* correcting some mistakes
* Adding tests for policies in Dp
* fixing a testname
* Updating the dataplane mock file
* removing dataplane mocks from dataplane tests as their scope is controllers
* gofumpt
* fix unsafe map access
* add metric wrapper to restserver IP endpoints
* add metrics to ip churn
* add metrics for ip pool scaling latency
* address review comments
* make MetricsBindAddress configurable
* use ifaces and test alloc metrics
* rename alloc to pool and update usage of scaler metric
* address review comments
* write to temp file and move to state file
* fixed memleak and other issues
* call windows replace function with MOVEFILE_WRITE_THROUGH flag
* moved few functions to platform package
* moved test files to correct dir
* addressed comments
* changes to support http and https for CNS
* correct small typo
* address feedback
* address feedback
* rename file
* small fix
* small fix
* fixing issue with cert not found
* initial changes for CNS->DNC support
* Adding changes for CNS to be compatible with managed DNC (reverse communication channel)
* adding NC version validation with respective NMA
* return errors for respective NC based on orchestrator context from CNI
* add nc version check via NMA
* adding logic to SyncNodeStatus and check if NCWaitingForUpdate for CniADD and CnsAttach calls
* addressing most of ashvin's comments
* adding managed config
* fat rebase
* addressing some comments
* slight optimizations...
* adding channel mode instead of managed bool
* set err in register node so that we keep looping
* addressing ashvin's comments
* fix test
* removing swift prefix mods for mdnc
* addressing tamanoha's comments
Co-authored-by: Jaeryn <tsun.chu@microsoft.com>
* About to take debugging comments out
* Removed debugging statements and tested again
* Removed more debugging statements and unused method
* Made changes suggested by Tamilmani, tested again.
* Made spacing changes suggested by Tamilmani
* Triggering new tests to run
* Added AITelemetry support for CNI
* added new files
* added other configs in config file
* fixed ut
* updated disableall similar to cns
* added container name to report
* addressed review comments
* addressed review comments
* added check for azure environment
* added log
* close log handle in unit test
* addressed review comments
* addressed review comments
* fixed a condition
* keep the netagent channel for logs
* fixed error
* addressed review comments
Expose 2 APIs to publish and unpublish network containers from CNS.
a. PublishNetworkContainer
b. UnpublishNetworkContainer
DNC calls CNS to publish and unpublish the network containers.
This PR adds support for host NC bidirectional communication with windows HnsV2. This is supported in multitenant scenario only. AllowHostToNCCommunication and AllowNCToHostCommunication flags are used to enable Host to NC and NC to host communication respectively.
* added application insights telemetry package to send metrics and logs to appinsights
* moved a function common to both packages
acquired a lock before writing to file
* added read write lock as per chandan comment. Addressed jaeryn comments
* fixed telemetry unit test
* defined interface and added appinsights package as vendor to acn
* added vendor package.go(appinsights)
* dependencies of appinsights
* added AI dependencies
* updated unit tests
* addressed review comments
CNM starts by default when CNS starts. This change prevents CNM
from starting by default. -startcnm command line option can be used
to start CNM if needed.
* configure dns server on azure bridge for ubuntu versions > 18. Moved hns related apis from platform package to hsnclient package
* read dns server from interface instead of file and fixed few issues
* renturn error on fail to configure dns
* added unitest and fixed an issue in ReadFileByLines method
* added a UT
* addressed comments and added few tests to get code coverage
* confiure dns for ubuntu17 also
* corrected typo error of filename
* Add option in CNS to pre-provision hns network
Add a commandline option in CNS to pre-provision hns network.
The commandline option take the type of the network that needs
pre-provisioning. This allows orchestrators to start CNS with this
option so that the VM network blip / disconnect is avoided when
calling cni add the very first time.
* Pass CNS server url when invoking CNI during add/del call in windows and update call in linux.
* Addressing Tamilmani's comments.
* Forgot to fix linux side of networkcontainers_linux.go
* Enabling exact match for pod name in attach/detach containertonetwork APIs.
* Returning the correct error when invoking Cni.
* Checking if orchestrator is Batch for attach/detach network calls.
* Moving similar lines for NewNetPluginConfiguration into a single function.
* Removing CNI configuration manipulation in CNS.
* Addressing Tamilmani's comments.
* Addressing Vivek's comments.
* Addressing Tamilmani's comments.
* Addressing Vivek's comment.
* Addressing Tamilmani's comments.
* Spawn telemetry buffer in a separate process instead of goroutine.
* Adding an option to disable telemetry.
* Addressing some of Tamilmani's comments.
* 1. Start telemetry service as separate process for all components cns/networkmonitor
2. Added telemetry config through which we can configure reportTohost interval
3. Added unit tests for changes
* added new files for telemetry testing
* added tests for invalid cases
* updated with dummy subid
Alexander
Paul Yu
Jackie Luc
Behzad Mirkhanzadeh
Saksham Mittal
estebancams
Matthew Long
Hunter Gregory
Mathew Merrick
Quang Nguyen
Quang Nguyen
tamilmani1989
Ramiro
aegal
Paul Johnston
Vamsi Kalapala
Eng Zer Jun
Evan Baker
tamilmani1989
Jaeryn
vivekagg-MSFT
Ashutosh Dubey
Ashvin Deodhar
Ashvin Deodhar