Граф коммитов

6 Коммитов

Автор SHA1 Сообщение Дата
Vikas Bhansali e02acffd38
Copyright fix (#1025)
* Correcting copyright statement
2023-01-07 08:11:07 +05:30
Vikas Bhansali cb4d5445ed
MD5 Sum updation on upload and validation on download (#863)
* Testing migration to 1es hosted pool

* remove indent

* Agent Image

* Remove demands

* Set imageName

* Add correct demand

* Just echo hello world

* Correct indent

* remove the quotes =

* lets try this

* maybe this works

* yeet

* added quotes around the pool name, added agent name demand

* added vmImage name?

* illegal node name?

* hostedAgent = true

* Remove Agent Name

* checkout none

* simple yaml

* try other pool

* no quotes

* UPdate

* update tab

* checkout ne

* Try blobfuse pool

* Added steps to pipeline nightly test

* Fixed indent

* Remove checkout: none

* Add distro test

* fixed indent

* remove checkout

* Fix checkout

* fuse3

* Added RHEL 8

* Override image

* Install

* Added Centos

* Added Oracle

* Added debian

* Added Mariner and SUSE

* Added C build

* Added git to Deban

* Try to get centos 8 to work

* try another cond

* conditional print

* conditional print

* agent condition try 2

* try with variable

* Update OS mirrors

* Fix sed command

* Add to yum

* try removing -e

* no bedst

* Conditional script

* Add more if script

* fix centos 7 stuff

* add flags

* fix check

* tdnf with y option

* Added a stdio lib

* try to fix oracle 8.1 and centos 8.5

* RHEL add flags

* remove ski pborken

* try debian 10

* added build essentials pacakge

* try another rhel

* also install fusermount

* added whoami

* added groups

* trying to run as root

* trying to not run as root

* install fuse

* mariner build essential

* try to use yum to install git

* Add -y

* Get sources list

* remove suse

* spell sources

* find the file that looks for microsoft repos

* remove azure list

* add to debiam

* INstall fuse lib

* try to fix debian 11

* Added MSI test

* reenable suse

* Install cmake and gcc on rhel

* Adding cmake install step for MSI

* Failover methods are not supported in EL8 so remove that confi from default yum repo config file

* install cmake on suse

* Correcting config file name in MSI ste

* Manually install fuse3-devel on SUSE

* Ignore gpg checks in suse to allow installing fuse3-devel

* Correcting suse install step

* Install fuse-libs on rhel vms

* user fusermount3 to unmount as we are not installing fuse on all of the vms

* Trying rhel 7.3 image

* Add RHEL 7.5

* Try to install rh-python

* Trying to install libfuse in RHEL

* sudo for echo

* Use sudo correctly

* add -e

* Updated container and python package for rhrl

* Try to get RHEL 8 to work

* Try to use vault 8 repos

* try mirror

* try without a mrror

* try with enable epel

* add flags

* Get all RHEL to work

* Get rhel 7.5 to work

* Added Ubuntu nightly

* added params

* Use msft managed

* indent

* Use normal nightly pipeline

* Added Blob STE retry policy

* Add credential policy

* Added Datalake retry policy

* Correcting UT compilation failure

* Ignore data validation test on distros other than ubn

* Change retry policy factory name for datalake

* Reducing cache config for streaming test

* Remove data validation test from different auth options

* Remove quick_test as default is anyway true

* Move data validation as a seperate stage

* Move data validation as a seperate stage

* Move data validation as a seperate stage

* Add data validation as a configurable step

* Make exhaustive test disable most of the test cases

* Adding support for md5sum validation and updation

* Add tags for data validation builds

* Try running fuse2 on ubn20 for data-validation

* Try running fuse2 on ubn20 for data-validation

* Merge all git related test cases to save time

* Remove ubn18 from pipelines, and use ubn20 for fuse2 testing

* Use ubn18 for release pipeline as glibc version error is coming on other distros due to this

* Remove clone and e2e test from release pipelines

* Adding basic e2e checks as part of ci

* Undo e2e from ci as multiple ci can run in parallel and running e2e may make it flaky

* Updating changelog for retry policy

* Enable git and tar test cases in nightly data validation test

* Enable git tests in verbose tests

* Remove clone test from verbose test

* Correct code lint command

* Ignore fuse3 test wrapper as well in linting

* Have two different flags for setting and validation md5 sums

* Correct code linting issue

* Adding UT for various combinations of MD5

* File cache shall delete the local file if download fails

* Adding few test cases to azstorage utils

* Remove dead code

* Resolve UT issue with logging

* Use 1ES for ubn-18 in artifact pipeline

* Remove redundancy

* Add dependency

* Add dependency

* Move code coverage to 1ES but remove ubn-22

* Move code coverage to 1ES but remove ubn-22

* Add account cleanup step to code coverage

* Add cleanup stage in code coverage

* Add cleanup stage in code coverage

* Correcting work dir path

* correction as per review comments

* Always run account cleanup at the end

* install go before doing account cleanup

* Make coverage and account cleanup a configurable step

Co-authored-by: Gauri Prasad <gapra@microsoft.com>
2022-08-25 16:49:56 +05:30
Sourav Gupta 35cf5ff4a2
Module name fix (#847)
* Correcting module name

* Changing module name

* Adding output binary name in go build
2022-07-27 12:19:36 +05:30
Gauri Prasad 15b68d598a
Fixed some codespell errors (#745)
Co-authored-by: Gauri Prasad <gapra@microsoft.com>
2022-03-23 11:00:13 -07:00
Vikas Bhansali 1aea8449ba
Perf and Memory Optimizations (#704)
* First commit of blobfuse v2 code

* Adding hadle optimization for v2

* Making direct read a configurable option, disabled by default

* Rearrange structures to reduce padding and save memory

* Correcting attr-cache UT for flag changes

* Merging file-cache-test

* Make flag check a function

* Adding comments to handle methods

* Adding more fuse options for optimization

* Adding more fuse options for optimization

* Removing exectime from red flows

* Correcting profiling code to work for both cpu and memory

* Cleanup old pprof reports

* Adding error log for policy local file removal case and adding 10mil max file limit in attr-cache

* Correcting file-cache UT for notInCache cases

* Correcting libfuse UT to get handle object out of open return parameters

* Addressing review comments

* Removing 7.5 old curl from nightly as it does not make sense for v2 to test different curl versions

* Tuning fuse parameters for better perf

* Fuse parameter tuning

* Removing some fuse options as mount is failing on some platforms

* Exit pipeline stage if mount failed

* Populate fuse connection flag based on kernel capabilities

* Log kernel fuse capabilities and the one chosen by blobfuse on given platform

* Fail pipeline if blobfuse2 binary is not found in df output

* Wait for some time for mount to go through before validating

* Correcting validation of mount point

* Correcting validation of mount point

* Use fuse2 on Deb-10 as with fuse3 we are not able to mount on that distro

* Update mount validation for huge listing

* Image signing to happen only when release option is chosen

Co-authored-by: Gauri Prasad <gapra@microsoft.com>
Co-authored-by: Ubuntu <vibhansa@vibhansa-perftest.l1vcg1e4u55e3gwz1dsfkxy2if.tx.internal.cloudapp.net>
2022-02-19 15:56:43 +05:30
Gauri Prasad 990af0fefa First commit of blobfuse v2 code 2022-02-14 09:41:31 -08:00