* parsing os args
* Removing auto completion
* Update changelog
* Add more flags to ignore
* Small change'
* coverage edit
* Removing dead code
* Add test cases for parse and validate config
* Init logger for UT on config
* Correctly define const slice for fuse ignored flags
* Adding unmount test cases
* Add comments as per review comments
* Correcting unmount tests
* Add logs if mkdir fails
* spell correction in comments
* Run all unmount tests serially with different mount points
* correcting unmount test cases
* Making unmount test parallel
* Adding handle map test cases
Co-authored-by: vibhansa-msft <vibhansa@microsoft.com>
* Testing migration to 1es hosted pool
* remove indent
* Agent Image
* Remove demands
* Set imageName
* Add correct demand
* Just echo hello world
* Correct indent
* remove the quotes =
* lets try this
* maybe this works
* yeet
* added quotes around the pool name, added agent name demand
* added vmImage name?
* illegal node name?
* hostedAgent = true
* Remove Agent Name
* checkout none
* simple yaml
* try other pool
* no quotes
* UPdate
* update tab
* checkout ne
* Try blobfuse pool
* Added steps to pipeline nightly test
* Fixed indent
* Remove checkout: none
* Add distro test
* fixed indent
* remove checkout
* Fix checkout
* fuse3
* Added RHEL 8
* Override image
* Install
* Added Centos
* Added Oracle
* Added debian
* Added Mariner and SUSE
* Added C build
* Added git to Deban
* Try to get centos 8 to work
* try another cond
* conditional print
* conditional print
* agent condition try 2
* try with variable
* Update OS mirrors
* Fix sed command
* Add to yum
* try removing -e
* no bedst
* Conditional script
* Add more if script
* fix centos 7 stuff
* add flags
* fix check
* tdnf with y option
* Added a stdio lib
* try to fix oracle 8.1 and centos 8.5
* RHEL add flags
* remove ski pborken
* try debian 10
* added build essentials pacakge
* try another rhel
* also install fusermount
* added whoami
* added groups
* trying to run as root
* trying to not run as root
* install fuse
* mariner build essential
* try to use yum to install git
* Add -y
* Get sources list
* remove suse
* spell sources
* find the file that looks for microsoft repos
* remove azure list
* add to debiam
* INstall fuse lib
* try to fix debian 11
* Added MSI test
* reenable suse
* Install cmake and gcc on rhel
* Adding cmake install step for MSI
* Failover methods are not supported in EL8 so remove that confi from default yum repo config file
* install cmake on suse
* Correcting config file name in MSI ste
* Manually install fuse3-devel on SUSE
* Ignore gpg checks in suse to allow installing fuse3-devel
* Correcting suse install step
* Install fuse-libs on rhel vms
* user fusermount3 to unmount as we are not installing fuse on all of the vms
* Trying rhel 7.3 image
* Add RHEL 7.5
* Try to install rh-python
* Trying to install libfuse in RHEL
* sudo for echo
* Use sudo correctly
* add -e
* Updated container and python package for rhrl
* Try to get RHEL 8 to work
* Try to use vault 8 repos
* try mirror
* try without a mrror
* try with enable epel
* add flags
* Get all RHEL to work
* Get rhel 7.5 to work
* Added Ubuntu nightly
* added params
* Use msft managed
* indent
* Use normal nightly pipeline
* Added Blob STE retry policy
* Add credential policy
* Added Datalake retry policy
* Correcting UT compilation failure
* Ignore data validation test on distros other than ubn
* Change retry policy factory name for datalake
* Reducing cache config for streaming test
* Remove data validation test from different auth options
* Remove quick_test as default is anyway true
* Move data validation as a seperate stage
* Move data validation as a seperate stage
* Move data validation as a seperate stage
* Add data validation as a configurable step
* Make exhaustive test disable most of the test cases
* Adding support for md5sum validation and updation
* Add tags for data validation builds
* Try running fuse2 on ubn20 for data-validation
* Try running fuse2 on ubn20 for data-validation
* Merge all git related test cases to save time
* Remove ubn18 from pipelines, and use ubn20 for fuse2 testing
* Use ubn18 for release pipeline as glibc version error is coming on other distros due to this
* Remove clone and e2e test from release pipelines
* Adding basic e2e checks as part of ci
* Undo e2e from ci as multiple ci can run in parallel and running e2e may make it flaky
* Updating changelog for retry policy
* Enable git and tar test cases in nightly data validation test
* Enable git tests in verbose tests
* Remove clone test from verbose test
* Correct code lint command
* Ignore fuse3 test wrapper as well in linting
* Have two different flags for setting and validation md5 sums
* Correct code linting issue
* Adding UT for various combinations of MD5
* File cache shall delete the local file if download fails
* Adding few test cases to azstorage utils
* Remove dead code
* Resolve UT issue with logging
* Use 1ES for ubn-18 in artifact pipeline
* Remove redundancy
* Add dependency
* Add dependency
* Move code coverage to 1ES but remove ubn-22
* Move code coverage to 1ES but remove ubn-22
* Add account cleanup step to code coverage
* Add cleanup stage in code coverage
* Add cleanup stage in code coverage
* Correcting work dir path
* correction as per review comments
* Always run account cleanup at the end
* install go before doing account cleanup
* Make coverage and account cleanup a configurable step
Co-authored-by: Gauri Prasad <gapra@microsoft.com>
* Remove dead code and fix file-cache usage percentage check UT
* Add coverage test for gen-test-config
* Adding code coverage for doc command
* Adding build script to build with different options
* Delete old binary before regenerating
* Delete old binaries before regenerating
* Adding pipeline test cases
* Remove agent cleanup steps as 1ES is spawning new vms for every run
* Disable sas validation on ubn-22 for now
* Disable sas validation on ubn-22 for now
* Disable sas validation on ubn-22 for now
* Removing ubn-22 from all pipelines
* Correcting adls sas param
* Re-add sas validation for all ubn distros
* Disable chmod test cases for SAS auth
* Set default value for sas to be false
* Converting account sas to container sas for chmod test to work
* Enable e2e test again
* Correcting e2e failure with chmod test case
* Add Ubn-22 to release pipeline
* Change pool of ubnuntu to ms hosted
* Remove dead param from yaml file
* Adding Ubn-22 to pipelines
* Adding lint as standard ci step
* correcting lint command path
* Add copyright checks to ci pipeline
* Copyright fixes and ci check for copyright statements
* Adding go format check to CI
* Fixing some lint issues
* More lint fixes
* More lint fixes
* Fixing some lint issues
* Correcting code linting check in ci
* correcting lint status check
* correcting lint status check
* correcting lint status check
* correcting lint status check
* correcting lint status check
* correcting lint status check
* correcting lint status check
* correcting bugs reported by sa check
* notices_fix.sh update
* Add step to validate NOTICE file
* Add step to release for ubn-22
* Add logs for success
* Fixing copyright bug
* Correcting tag formats
* Correcting percentage check
* Spell error correction
* Ignore exectime from code coverage
* Correcting log string
* Removing dead file
* Correcting go format error
* Fixed df on blobfuse incorrect stats
* Fixed df on blobfuse incorrect stats
* cleanup
* added test
* added test
* cover the case of exceeding cache
* case with unlimited file cache
* file cache does fstat
* rename
* merge main
* Adding more fuse options for optimization
* Correcting logs for fuse3
* Removing nullpath feature as fuse3 does not support it
* Remove symlink cache as fuse3 does not support that
* Convert expected err log to warn
* Removing dead code
* Try file read using native C api
* Try early pread to improve perf
* Adding dynamic profiler and disable native pread call
* Log error when profiler fails to start
* Try prefetching data into memory for disk-file
* Make dynamic profiler available in both foreground and daemon mode
* Move read/write call to native calls
* Add call to update the file-cache when native read/writes are invoked
* Remove cache updating call for better perf
* Remove readahead logic from file-cache
* Clear handle map dead code
* convert fd to uint64 instead of int
* Adding basic infra for read ahead in native code
* Clean up code, remove read-ahead logic and make direct read the default option
* Correcting the flag name for offload-io
* Correcting spell mistakes and adding comments
* Correcting fsync test failure
* Correct fuse3 UT for fsync and handle conversion
* Resolve compilation failure for filehandle struct
* Add logic to update the cache on every 1K operations on a handle
* Resolve failure with flush when native write is enable
* Add read-ahead logic in native code
* Compilation failure resolution on fuse2
* Correcting memset params
* Correcting spell error
* Add GOGC and MAXPROCS value for perf tuning
* Reduce min file size and correct read ahead buffer size for smaller fiels
* Replace fsync with close of duplciate handle
* Correcting write flow file corruption issue
* Handle write in file-cache instead of libfuse as the other approach is corrupting data somewhere
* File-cache to refresh cache on eveyr 1K operations and data validation test to wait for 2 seconds after copy
* Replace file-cache read/write calls with syscalls
* Enable native read/write again with read ahead disabled
* Remove cache update from read calls
* Remove read ahead logic completly
* Cleanup libfuse compo and e2e test changes
* Delete emptry directories recursively till root
* Changes as per review comments
* Fix issue with block-size running into overflow of number of blocks
* Add blocksize and file size to logs
* Make blocksize function to work on file size instead of fi object and add UT for blocksize calculations
* Adding more ut case
* Add some breather between upload and test
* FileCacheTimeout test case to wait untill file is deleted from local system
* FileCacheTimeout test case to wait untill file is deleted from local system
* Codespell error fix
* Restructuring code as per review comments and internal discussions
* Code spell fixes
* Stop updating cache on read
* Adjusting cache update operation counter
* Correcting libfuse_read flow and adding option to enable offload option manually
* Correcting file_cache UT configs
* Adding dynamic profiler config in base config
* Added metadata to handle to attempt to preserve metadata. WIP
* Added metadata preservation feature to attribute cache
* Removed test from file test
* Fixed unit tests
* Added code to ensure metadata is retreived
* metadata preservation
Co-authored-by: Gauri Prasad <gapra@microsoft.com>
* Correcting UT for disk cache
* Sync with main and review comment addressed
* Keeping default persisted block timeout to 30 mins
* changes as per review comments
* adding comments to UT for disk-persistence
* Copyright info update
* First commit of blobfuse v2 code
* Adding hadle optimization for v2
* Making direct read a configurable option, disabled by default
* Rearrange structures to reduce padding and save memory
* Correcting attr-cache UT for flag changes
* Merging file-cache-test
* Make flag check a function
* Adding comments to handle methods
* Adding more fuse options for optimization
* Adding more fuse options for optimization
* Removing exectime from red flows
* Correcting profiling code to work for both cpu and memory
* Cleanup old pprof reports
* Adding error log for policy local file removal case and adding 10mil max file limit in attr-cache
* Correcting file-cache UT for notInCache cases
* Correcting libfuse UT to get handle object out of open return parameters
* Addressing review comments
* Removing 7.5 old curl from nightly as it does not make sense for v2 to test different curl versions
* Tuning fuse parameters for better perf
* Fuse parameter tuning
* Removing some fuse options as mount is failing on some platforms
* Exit pipeline stage if mount failed
* Populate fuse connection flag based on kernel capabilities
* Log kernel fuse capabilities and the one chosen by blobfuse on given platform
* Fail pipeline if blobfuse2 binary is not found in df output
* Wait for some time for mount to go through before validating
* Correcting validation of mount point
* Correcting validation of mount point
* Use fuse2 on Deb-10 as with fuse3 we are not able to mount on that distro
* Update mount validation for huge listing
* Image signing to happen only when release option is chosen
Co-authored-by: Gauri Prasad <gapra@microsoft.com>
Co-authored-by: Ubuntu <vibhansa@vibhansa-perftest.l1vcg1e4u55e3gwz1dsfkxy2if.tx.internal.cloudapp.net>
Vikas Bhansali
Sourav Gupta
ashruti-msft
Abhinav Gupta
abhiguptacse
Tamer Sherif
Gauri Prasad
Tamer Sherif
Gauri Prasad