5b2e0d31de
support readonly mount
...
fix unittests
2017-08-07 16:27:47 -07:00
f07c7f7cf9
Address comments
2017-07-31 13:13:05 -07:00
1504693df8
Reuse the client for openread
2017-07-31 13:13:05 -07:00
6d998fe4d3
Closing unused connections. ( #27 )
...
Closing the connections to namenode after finishing or failures.
2017-06-22 17:52:27 -07:00
63723f39b9
Add version and built info
...
Use current GIT commit ID and built info (time and machine) for versioning purpose
2017-06-21 17:27:51 -07:00
1096040f83
StatFs test for FileSystem's StatFs
2017-06-20 21:49:05 -07:00
d637983f48
ZipFile: Add logging if error opening Zipfile
2017-06-20 21:49:05 -07:00
b903bd2957
ZipFile: Tests for functions with zipfile
2017-06-20 21:49:05 -07:00
249e738b2b
FileHandleWriter_test: Test file flushing
2017-06-20 21:49:05 -07:00
1820e99270
Allow Travis failure with Go-1.5 in case of some uncompatible APIs
2017-06-20 21:49:05 -07:00
f184a3b20d
Add skipped test for overwriting existing file in FileHandleWriter
2017-06-20 21:49:05 -07:00
6ed81c7fc8
Add code coverage report file for codecov
2017-06-20 21:49:05 -07:00
e02774add0
Refactoring: fix the generated executable filename
2017-04-17 12:04:53 -07:00
93458f0f4e
Makefile: collect the test coverage info
...
Reformat Makefile and add test coverage report
2017-04-17 12:04:46 -07:00
b50bd97baf
Add License info to files
2017-04-17 10:00:32 -07:00
d27e9f8cc6
Travis: Add sudo privilege to create stagedir under /var
2017-04-14 17:57:59 -07:00
3640223220
FileHandleWriter: Add error handling to stagedir creation
2017-04-14 17:57:59 -07:00
c184f41f10
Add test for FileHandleWriter
2017-04-14 17:57:59 -07:00
d6a9ff305e
Remove the MaxFileSizeForWrite limitation
...
Query the HDFS remaining size to write
2017-04-14 17:57:59 -07:00
1b8cef36a5
Chown support ( #18 )
2017-03-28 10:33:24 -07:00
b85f319dc3
Increase the maximum file handler from 1K to 1M ( #16 )
...
The default file handler limitation on Linux for each process is 1K, which could be too few. Increase this number to 1M for hdfs-mount process only.
This option requires sudo privilege, or no change happens with an error msg.
2017-03-17 09:19:19 -07:00
e4218b854f
Travis CI: add Go-1.7 version
2017-03-15 16:59:10 -07:00
db9b27eed6
Update submodule to latest master
2017-03-15 10:20:07 -07:00
fee196f057
Leveled log information ( #15 )
...
* Leveled log information
New option "-logLevel=<int>" for hdfs-mount for different log levels.
-logLevel=0, default, print out error logs only
-logLevel=1, print out error and warning logs
-logLevel=2, print out all logs
* Change some logs from error to warning, as long as the basic functions finish correctly
2017-03-10 14:31:38 -08:00
8889acd884
Fix for seek error in overwriting an exsiting file ( #14 )
...
While overwriting the exsiting file, hdfs-mount tries to read the exsiting file first. This ocassionly causes I/O warning that seek-offset == file-offset, and OS takes the I/O warning as error.
The fix is not to report error when seek-offset == file-offset.
2017-02-21 10:55:20 -08:00
c5b6c5b4a7
FileHandleReader: Correct log information on offset
2016-12-28 15:01:42 -08:00
5a1e7c9969
Temporally increase the limit for file size to 200GB
2016-11-29 14:43:12 -08:00
21ee4484b7
Update Ci info in README
2016-10-27 12:29:39 -07:00
8aeb131e78
hdfs permission mode: add chmod support in hdfs fuse ( #9 )
...
1. use SetattrRequest in bazil.org/fuse for permmode
2. test setattr in Dir_test.go
2016-10-27 12:27:11 -07:00
c7f326ac90
Add travis CI support
2016-10-26 13:22:00 -07:00
538db9865f
Update submodule to latest master
2016-10-26 13:17:38 -07:00
fd573e87d7
Update submodule to latest master
2016-09-28 14:13:55 -07:00
5a49eee321
Update .gitmodules
2016-09-28 14:05:27 -07:00
797e92e600
Bugfix for renaming files/dirs. Ensuring that in-memory file entry is in sync with rename
2016-02-20 21:15:30 -08:00
e5aba004b3
Experimental write support (merging vlivan/staged-writes branch)
...
This change contains experimental support for write operations.
Due to lack of some features in underlying HDFS client library
(work on that in progress) this change temporarily implements
staged writes approach: files for writing are buffered on local disk
and flushed to HDFS by uploading full file content.
This change supports random writes and appends (arbitrary seek&write patterns),
however Flush() operation is super-slow, since uploading to HDFS
is happening during Flush() only, or at Close() because of implicit Flush
Being experimental this change also contains some hardcoded values
(like faking fstat request,m ax supported file size for writing, replication
factor = 3, etc).
Those values will be configurable in future or will be removed
2016-01-29 16:40:39 -08:00
7ca4e505a2
Makefile: ensuring that repo cloned recusively before building
2016-01-23 21:28:30 -08:00
816934690f
Switching to Go 1.6beta2 and switching to use private fork of colinmark/hdfs to track necessary bugfixes
2016-01-23 21:16:53 -08:00
01c9708c82
Implementing fix for connection leak and reverting previous GC workaround
2016-01-23 01:30:37 -08:00
f66cb20e41
Merge branch 'vlivan/zip-performance' - Improving zip files access performance
...
by order of magnitude
2016-01-20 17:28:40 -08:00
838e9859aa
Minor refactoring, renaming fields to remove disambiguity
2016-01-20 17:19:16 -08:00
463e116875
Fix for proper initialization of fields of zip nodes
2016-01-20 17:16:27 -08:00
0330c96b5a
Follow-ups for CR from alexeyk
2016-01-20 16:08:39 -08:00
773590ee15
Using FileHandleReader as a backend data provider for RandomAccessReader instead of HdfsReader. This allows to leverage all optimizations for sequential reads for zip archives
2016-01-20 15:45:20 -08:00
47e4473d74
Refactoring: randomAccessHdfsReaderImpl -> randomAccessReaderImpl
2016-01-19 22:13:49 -08:00
bebb5d44f9
Added FileHandleAsReadSeekCloser - a ReadSeekCloser adapter to FileHandle
2016-01-19 20:54:55 -08:00
e472a61a51
Refactoring: RandomAccessHdfsReader -> RandomAccessReader
2016-01-19 20:32:36 -08:00
48de17976b
Refactoring: MockPseudoRandomHdfsReader -> MockReadSeekCloserWithPseudoRandomContent
2016-01-19 20:27:35 -08:00
72fe16c7f6
Refactoring: hdfsReaderImpl -> HdfsReader
2016-01-19 20:24:46 -08:00
de9eb65685
Refactoring: HdfsReader interface -> ReadSeekCloser
2016-01-19 20:23:07 -08:00
72dbf23f38
io.ReadFull instead of reader.Read() for ReadAt interface
2016-01-19 18:33:56 -08:00
clchen
Junjie Qian
Paresh Verma
Junjie Qian
Marko Radmilac
Marko Radmilac
Vladimir Ivanov