Граф коммитов

1148 Коммитов

Автор SHA1 Сообщение Дата
Mathew Merrick 4658dfd395
retry ci (#1134) 2021-11-30 13:57:54 -06:00
Mathew Merrick 3dbb992c50
chore: create dependabot.yaml (#1135) 2021-11-30 13:57:34 -06:00
Hunter Gregory 74c79f701b
test: [NPM] Profiles (#1104)
* v2 profile (with azure chain first) and v1 profile for both chain positions

* disable pprof and remove chain placement toggle for v2

* parallelize cyclonus tests for multiple NPM profiles

* rename things and try 2 different profiles run in parallel

* update v2 toggle name in profiles

* remove v2 cyclonus test and add one for placing azure chain first (v1)
2021-11-30 10:55:42 -08:00
Matthew Long 0c3efef7ff
feat: nnc client sets ownerref (#1133)
* feat: nnc client sets ownerref

* fix: linter issues

* fix: don't use unnecessary custom error type
2021-11-29 19:49:55 -06:00
Evan Baker c8014bc097
update build tools (#1127)
Signed-off-by: Evan Baker <rbtr@users.noreply.github.com>
2021-11-29 19:05:32 -06:00
Evan Baker a664889867
fix IPs typos in CNS code and comments (#1111)
Signed-off-by: Evan Baker <rbtr@users.noreply.github.com>
2021-11-29 15:48:31 -06:00
tamilmani1989 82f4b391a1
feat: [CNI] Enable DSR policy for windows singletenancy (AKS) by default (#1116)
* added dsr changes for windows

* fixed lint and added unit test
removed unused error

* skip adding dsr policy for hnsv1

* addressed comments
lint fix

* fixed windows uts
2021-11-29 10:12:32 -08:00
Evan Baker 7c5cdde1b8
make hook install optional (#1131)
Signed-off-by: Evan Baker <rbtr@users.noreply.github.com>
2021-11-24 15:54:20 -06:00
Vamsi Kalapala 303e237688
fix: adding relevant platform suffix to ignore errors in windows (#1119) 2021-11-23 09:18:17 -08:00
Mathew Merrick 7b8ad95cec
chore: update windows npm to servercore base image (#1122) 2021-11-22 16:36:40 -08:00
Evan Baker 7fbbd1e0b7
lint pre-push hook (#1112) 2021-11-22 12:56:58 -06:00
Evan Baker 22906eb9b2
fix ipampool scaling - only release IPs we have (#1110)
Signed-off-by: Evan Baker <rbtr@users.noreply.github.com>
2021-11-22 12:22:30 -06:00
Paul Johnston 6d208e9762
Cns windows aks (#1059)
* chore: add in some functionality for CNS on windows host process pods
2021-11-22 09:23:01 -08:00
Vamsi Kalapala ca3457e5ee
fix: [NPM] adding fixes to windows DP after first run (#1100)
* [NPM] Expanding HNS fake usage with internal state

* adding some test cases for windows DP

* adding some test cases for windows DP

* Correcting some issues with windows DP

* Splitting each modify call into two,  one for 1st level sets and another for nested sets

* Fixing a build issue in linux

* Enhancing windows ipset tests

* Adding ipset mgr windows tests

* fixing a build issue

* Adding in ACL verify code

* addressing some lint issues

* addressing some lint issues

* removing apply ipset in generic ipsetMgr tests

* fixing a build issue with windows

* fixing windows build issue
2021-11-19 14:57:38 -08:00
JungukCho bba5352d66
feat: [NPM] Update feature toggles and corresponding codes for enabling V2 NPM test (#1115)
* Update codes to enable V2 NPM

* Deleted dead codes (if we want to keep it, please let me know)

* Update azure-npm.yaml to add toggle parameters

* Fix incorrect call for v2 NPM
2021-11-19 09:31:42 -08:00
Hunter Gregory 5d2da73cba
fix: [NPM] Testing Changes and updates to Linux Policy Manager (#1114)
* wip

* address comments and simplify logic a bit for getting current azure chains

* fix order of resetting acls and ipsets
2021-11-19 08:41:27 -08:00
Paul Johnston b0b6edfcfe
Azure cns yaml (#1058)
* [feat] making azure-cns-windows yaml
2021-11-18 12:29:51 -08:00
Hunter Gregory da4bd0d43b
feat: [NPM] Reset ipsets & update a Prometheus function ResetIPSetEntries (previously just used for UTs) (#1108)
* fix: for prometheus ResetIPSetEntries & feat: reset ipsets in NPM v2

* add note about difference in prometheus metrics in v1 vs v2, and strengthen a UT

* add comment to delegate prometheus metrics from generic ipsetmanager to OS-specific ones

* fix UT for dataplane_test.go and fix lint

* rename variables based on suggestions

* switch to unnamed return values (will throw a go lint error)
2021-11-18 10:00:54 -08:00
JungukCho 742e1fbe14
refactor: [NPM] Minor fixes (does not change functionality, but for better maintenance) (#1113)
* Minor fixes (does not change functions, but for maintanance)

* Reorganizes codes

* Address lint warning

* Resolve compile error by removing ICMP related codes in windows
2021-11-18 09:51:05 -08:00
JungukCho fa352e3a54
refacto: [NPM] Handle 0.0.0.0/0 ipblock and de-duplicated elements in except field (#1109)
* Handle 0.0.0.0/0 ipblock and duplicated elements in except field

* Update v1 as well and remove unneeded comments

* Address comments (reoganize UTs and except handler in dataplane)

* Format except cidr for better handling and updated UTs accordingly
2021-11-18 09:50:42 -08:00
Hunter Gregory 618a65447b
feat: [NPM] Clean up iptables chains in Linux v2 (#1090)
* cleanup old policy chains and reboot iptables chains when there are no more policies

* remove get prefix for all functions per junguks feedback

* clean up code for port specs and fix a lint

* address comments

* remove stop channel in OS-specific reconcile

* move policy methods to policy_linux.go

* add comments based on suggestions

* fix build issue:  move a constant from linux file to generic file
2021-11-17 15:04:11 -08:00
Eng Zer Jun e812bc82b8
refactor: move from io/ioutil to io and os packages (#1096)
The io/ioutil package has been deprecated as of Go 1.16, see
https://golang.org/doc/go1.16#ioutil. This commit replaces the existing
io/ioutil functions with their new definitions in io and os packages.

Signed-off-by: Eng Zer Jun <engzerjun@gmail.com>
2021-11-17 16:31:42 -06:00
JungukCho cd438069f5
Use Equal function for label instead of using reflect.DeepEqual in namespaceController (#1107) 2021-11-17 12:58:56 -08:00
Evan Baker 7843fb11ef
use temp goldpinger image which supports IPv6 and only pings ready pods (#1092)
Signed-off-by: Evan Baker <rbtr@users.noreply.github.com>
2021-11-17 11:22:20 -06:00
Hunter Gregory db3c7065bb
fix: [NPM] reposition iptables jump to AZURE-NPM chain (#1086)
* put jump from forward to azure-npm chain above the one to kube-services

* update unit test

* add toggle for chain position

* incorporate toggle in iptm and update UTs. v1 controller tests seem broken

* rename toggle name

* jump to azure chain on new ct state and update default toggle (UTs will break)

* make util constant for UTs and fix UT errors (besides ones I get for controllers)

* added missing module args for ctstate NEW

* reconcile jump to azure chain at top

* delete deprecated jump to azure chain on uninit, and fix go lint

* assign correct default toggle value

* addressed comments

* fix UTs after removing index 1 for placing chain first. Also make all tests subtests for check and add forward chain

* set PlaceAzureChainFirst: true

* switch to correct default for PlaceAzureChainFirst
2021-11-16 15:30:34 -08:00
Hunter Gregory 17ed0b83fa
feat: [NPM] ipset save before restoring and fix grep UTs (#1085)
* use ipset save to update members and update error handling logic for ipsets to skip previously run lines. Also update some logging for iptables chain management

* remove unused code

* add comment block describing high level ipset restore logic

* fix bug in piping to grep. need to add pipe errors for fexec UTs so that we dont revert on dataplane UTs

* grep for npm sets working for ipsets save, but this breaks DP UTs

* VerifyCalls method for mock ioshim

* add ability to unit test piped commands

* update logging

* UTs for piping a command to grep

* grep for npm sets in ipset save, verify number of calls in UTs, and update ApplyIPSets test calls for dataplane UTs

* update comments based on PR suggestions

* addressing comments

* remove out-of-scope policy changes for this PR

* rename restore file creator files

* FIXME: setting v2 controllers toggle to true to create an image in pipeline

* Revert "FIXME: setting v2 controllers toggle to true to create an image in pipeline"

This reverts commit 31148c3034.

* wrap errors
2021-11-15 17:47:05 -08:00
JungukCho d7dcc64f93
refactor: [NPM] General translation logic for egress and ingress (#1106)
* Generallize function for ingress and egress

* Use function to support both ingress and egress and update UTs

* Simplify UTs and correct comments

* Add more comments and correct policies.MatchType

* Reorganize codes to better understand and simplify UTs

* Addresses comments (e.g., fix typo and correct wrong comments)

* Address comments (e.g., srcList and dstList) and correct MatchType values

* Address comments (fromRulesExists -> peerRuleExists)
2021-11-15 17:38:08 -08:00
Evan Baker b1d7723c3a
use correct duration for pool refresh (#1094)
Signed-off-by: Evan Baker <rbtr@users.noreply.github.com>
2021-11-15 12:44:51 -06:00
JungukCho 0d2e1dc573
refactor: [NPM] General translation logic (mainly clean-up codes and correct bugs) (#1105)
* Delete unneeded codes for podSelector and update UTs

* Delete unneeded codes for nameSpaceSelector and UTs

* Delete unneeded codes in parseSelector

* Use nil slices instead of zero slices for TranslatedIPSet from namespaceSelector

* Use Variadic functions in NewTranslatedIPSet to use nil slice instead of empty slice and update UTs accordingly

* Use right settype for all-namespaces

* no export for flattenNameSpaceSelector function in parseSelector (vamsi's comments)
2021-11-12 13:46:50 -08:00
JungukCho 6835b8630f
refactor: [NPM] parsing label selector for general translation logic (#1077)
* Add comments for package and functions

* Handle namespaceSelector in simple way and remove unnecessary code for ipBlock field

* Translate podSelector in a simple way and update its UTs.

* Use parsedSelectors struct to reduce duplicated codes and update comments

* Address lint errors and add missing comments

* Correct comments

* Addressed comments

* Use correct settype for all-namespaces

* Address comments

* Addressed comments
2021-11-12 12:05:15 -08:00
JungukCho 76e7531a08
feat: [NPM] Support pod grace period for v2 (#1095)
* Use official k8s equals functions for label instead of our own implementation

* Update v2 pod controller to support graceful shutdown of pod
2021-11-10 16:01:21 -08:00
JungukCho cc68c342fa
feat: [NPM] Support graceful shutdown in pod (#1083)
* Support graceful shutdown in pod

* Update detailed comments and cleaning up codes

* Add Unit tests

* Address lint errors

* Address comments

* Addressed comment and add UTs
2021-11-09 17:24:42 -08:00
Vamsi Kalapala 08df7f7429
feat: [NPM] NPM v2 network policy controller and UTs for all v2 controllers (#1082)
* adding a legacy build command

* Adding all v2 controller test files

* v2 podcontroller changes

* completing all pod v2 controllers uts

* Adding netpol v2 controller UTs

* Removing unused make file command

* Fixing lints and correcting a test case

* Fixing an error in expected values

* dealing with flaky tests

* Fixing an issue with HCN vendor, until we wait for the fix to be rolled out

* Addressing some comments

* Removing addPolicy call and relying on updatepolicy

* Saving only spec of netpol and not whole object

* changing name of rawNPMap to rawNPSpecMap

* changing name of rawNPMap to rawNPSpecMap

* Deep equal type for spec was not equal corrected the pointers

* Deep equal type for spec was not equal corrected the pointers
2021-11-09 15:29:17 -08:00
tamilmani1989 cdab7d0241
Use Lockedfile api to acquire lock (#1070)
* added lockedfileapi support for CNI

* fixed interface changes

* addressed comments
fixed ut

* addressed comments

* fixed copy to buffer part in writer api

* fixed copy to buffer part in writer api

* keeping old code not changing it.
2021-11-09 08:19:44 -08:00
Vamsi Kalapala 077e11d192
[NPM] Fixing a bug in vendor files until we get an official build (#1091) 2021-11-08 11:46:59 -08:00
Hunter Gregory 6f26925d11
FIX: upload higher resolution npm v2 chain design image (#1072)
* delete low res image

* Add high res image

* make white background for image
2021-11-08 09:38:06 -08:00
Paul Miller 27ac431a6e
Use avast to retry init cns and register node. (#1087)
* stupid simple retry

* go lint fixes

* missed one //

* avast retry

* try out avast

* vendor

* try nad make linters happy

* fix wrap check
2021-11-05 13:20:26 -05:00
Mathew Merrick fa8f665ab5
ci: add windows build pool and NPM windows dockerfile (#1073)
* build pool windows
2021-11-05 10:30:09 -07:00
Vamsi Kalapala cb866ef956
[NPM] Moving v1 netpol controller into its new location (#1081)
* Mocing v1 netpol controller into its new location

* fixing lints

* Addressing some comments

* Changing var to const
2021-11-03 15:09:24 -07:00
JungukCho 8434e139d1
[NPM] General translation logic for linux and windows (#1055)
* Create generic translation struct and start using it networkPolicyController

* Update ipsm to get more information from NPMCache

* Working on translateIngress part and its UTs

* Done functions of Translate ingress rule (need to add UTs to test its functions and clean up codes)

* Use function for repeated codes

* Cleanup UTs

* Remove all unused codes in this PR except for ingress rules

* Create functions to make codes concise and reorganize ipsets for better readability

* Remove duplicated data for targetPod information in every ACL

* Move translation logics to /pkg/controlplane/translation dir

* Remove redundant codes and resolved some of lint errors

* Resolve lint errors and remove unused codes in parseSelector and its UTs

* Use unique id for acl policy among network policies and add UTs for port rules

* Addresses some comments (will resolve more later)

* Complete namespaceSelector UTs and correct some logics for handling namespaceSelector

* Use consistent variables and variable for flexibility in UTs

* Add more UTs for allowAll and defaultDrop rules and clean-up codes

* Remove unused codes

* Resolve lint errors

* Clean-up and reorganize codes

* Revert "Update ipsm to get more information from NPMCache"

This reverts commit 477bbaf43d56a6535f5cc035dfe15d5b6035647a.

* Address comments

* Resolve part of lint errors

* Add comments for todo things in next PR

* Delete unused file and clean-up code

* Fix Uts

* Remove unnecessary code
2021-11-01 09:03:16 -07:00
Vamsi Kalapala 66453dd20b
[NPM] Fixing a windows NPM build issue (#1074)
* [NPM] Fixing a windows NPM build issue

* building npm windows

* ignore building npm windows
2021-10-29 14:50:33 -07:00
Hunter Gregory 5490a9fa1f
NPM: Linux Policy Manager (#1069)
* feat: policy manager for linux

* remove composer from this PR (saving progress on my local machine)

* redesign iptables, rework/complete UTs

* use strings instead of constants in UTs

* fixed go lints

* fixed bug found in integration testing

* added integration tests for policymanager (should replace with dataplane interface calls later)

* rearrange UTs, add extra coverage for chain management, and fix bug for reporting error on chain destroy failures

* rename variable

* fixed lint (removed newline)

* update errors in policymanager and add an error in linux pMgr if we cant delete jump rules from ingress/egress chain to policy chain

* fix tiny lint

* address comments, update UTs in dataplane_test.go, update error wrapping, add windows UT files

* address more feedback and fix DP UTs by commenting out pMgr init/reset for now

* add comment
2021-10-29 13:57:19 -07:00
Vamsi Kalapala 9850758e4b
[NPM] Windows Policy Manager changes for OS22 (#1062)
* [NPM] Windows Policy Manager changes for OS22

* Adding new NPM ACLSettings with ID

* first pass on both add and remove policies

* fixing a merge issue

* Working 1st level Setpolicy CRUD operations

* have NPMACl to HNSACL conversion logic ready

* updating policy endpoints only after adding policy to an endpoint

* updating policy endpoints only after adding policy to an endpoint

* fixing a build issue

* fixing issue in linux files

* Addressing some comments and also completing some integrations with V2 control plane

* Updating policy ID logic and update pod

* Updating policy ID logic and update pod

* Addressing some comments

* adding basic reset bits

* fixnig build issue in linux

* Fixing the _linux_test.go build failures

* fix lints

* Addressing some comments and correcting windows logic to apply set policies in order

* cleaning up logic for calculating set policies

* Applying some feedback.

* fixing a failing test and panic
2021-10-28 08:50:41 -07:00
Hunter Gregory 2febacaf30
Upload Image: NPM v2 iptables chain design (#1071) 2021-10-27 15:37:10 -07:00
Mathew Merrick 7c2aa30b7a
ci: update e2e apimodel to 2004 (#1068)
* chore: update apimodel to 2004
2021-10-22 13:20:43 -07:00
Evan Baker 4a4370b0be
chore: tidy up nmagent client for context timeouts (#1056)
* chore: tidy up nmagent client for context timeouts

Signed-off-by: Evan Baker <rbtr@users.noreply.github.com>

* address review feedback

Signed-off-by: Evan Baker <rbtr@users.noreply.github.com>

* fix tests

* change integration test timeout to 1h

Signed-off-by: Evan Baker <rbtr@users.noreply.github.com>

* guard against nil in wireserver util

Signed-off-by: Evan Baker <rbtr@users.noreply.github.com>

* read body

Signed-off-by: Evan Baker <rbtr@users.noreply.github.com>

* correct durations

Signed-off-by: Evan Baker <rbtr@users.noreply.github.com>
2021-10-21 13:14:03 -05:00
Hunter Gregory 5a8931bc55
make exported testutils for ipsets (#1067) 2021-10-21 10:50:45 -07:00
Jaeryn 01085728bb
fix: Add NAT Policies for Windows AKS-Swift Scenario (#1036)
* Add DNS NAT policy for windows AKS-Swift scenario

* Addressing comments

Co-authored-by: Jaeryn <tsun.chu@microsoft.com>
2021-10-20 13:58:10 -07:00
Mathew Merrick 60820006e1
refactor: move existing npm controllers to v1 (#1040)
* put v2 behind toggle
2021-10-20 13:03:12 -07:00
Evan Baker 17bd9425d0
chore: tidy up the wireserver client and usage (#1065)
* chore: tidy up IMDS client

Signed-off-by: Evan Baker <rbtr@users.noreply.github.com>

* rename from imds to wireserver

Signed-off-by: Evan Baker <rbtr@users.noreply.github.com>
2021-10-20 14:23:05 -05:00