azure-kusto-parquet-conv/arrow-rs/CHANGELOG.md

Changelog

52.0.0 (2024-06-03)

Full Changelog

Breaking changes:

• chore: Make binary_mut kernel accept different type for second arg #5833 [arrow] (viirya)
• fix(flightsql): remove Any encoding of DoPutPreparedStatementResult #5817 [arrow] [arrow-flight] (erratic-pattern)
• Encode UUID as FixedLenByteArray in parquet_derive #5773 (conradludgate)
• Structured interval types for IntervalMonthDayNano or IntervalDayTime \#3125 \#5654 #5769 [parquet] [arrow] (tustvold)
• Fallible stream for arrow-flight do_exchange call \#3462 #5698 [arrow] [arrow-flight] (opensourcegeek)
• Update object_store dependency in arrow to 0.10.0 #5675 [parquet] (tustvold)
• Remove deprecated JSON writer #5651 [arrow] (tustvold)
• Change UnionArray constructors #5623 [arrow] [arrow-flight] (mbrobbel)
• Update py03 from 0.20 to 0.21 #5566 [arrow] (Jefffrey)
• Optionally require alignment when reading IPC, respect alignment when writing #5554 [arrow] [arrow-flight] (hzuo)

Implemented enhancements:

• Serialize Binary and LargeBinary as HEX with JSON Writer #5783 [arrow]
• Some optimizations in arrow_buffer::util::bit_util do more harm than good #5771 [arrow]
• Support skipping comments in CSV files #5758 [arrow]
• parquet-derive should be included in repository README. #5751
• proposal: Make AsyncArrowWriter accepts AsyncFileWriter trait instead #5738 [parquet]
• Nested nullable fields do not get treated as nullable in data_gen #5712 [arrow]
• Optionally support flexible column lengths #5678 [arrow]
• Arrow Flight SQL example server: do_handshake should include auth header #5665 [arrow] [arrow-flight]
• Add support for the "r+" datatype in the C Data interface / RunArray #5631 [arrow]
• Serialize FixedSizeBinary as HEX with JSON Writer #5620 [arrow]
• Cleanup UnionArray Constructors #5613 [arrow] [arrow-flight]
• Zero Copy Support #5593
• ObjectStore bulk delete #5591
• Retry on Broken Connection #5589
• StreamReader is not zero-copy #5584 [arrow]
• Create ArrowReaderMetadata from externalized metadata #5582 [parquet]
• Make filter in filter_leaves API propagate error #5574 [arrow]
• Support List in compare_op #5572
• Make FixedSizedList Json serializable #5568 [arrow]
• arrow-ord: Support sortting StructArray #5559 [arrow]
• Add scientific notation decimal parsing in parse_decimal #5549 [arrow]
• take kernel support for StringViewArray and BinaryViewArray #5511 [arrow]
• filter kernel support for StringViewArray and BinaryViewArray #5510 [arrow]
• Display support for StringViewArray and BinaryViewArray #5509 [arrow]
• Arrow Flight format support for StringViewArray and BinaryViewArray #5507 [arrow] [arrow-flight]
• IPC format support for StringViewArray and BinaryViewArray #5506 [parquet] [arrow]

Fixed bugs:

• List Row Encoding Sorts Incorrectly #5807 [arrow]
• Schema Root Message Name Ignored by parquet-fromcsv #5804 [parquet]
• Compute data buffer length by using start and end values in offset buffer #5756 [arrow]
• parquet: ByteArrayEncoder allocates large unused FallbackEncoder for Parquet 2 #5755 [parquet]
• The CI pipeline Archery test With other arrow is broken #5742 [arrow]
• Unable to parse scientific notation string to decimal when scale is 0 #5739 [arrow]
• Stateless prepared statements wrap DoPutPreparedStatementResult with Any which is differs from Go implementation #5731 [arrow] [arrow-flight]
• "Rustdocs are clean amd64, nightly" CI check is failing #5725 [parquet] [arrow]
• "Archery test With other arrows" integration tests are failing #5719 [parquet] [arrow]
• parquet_derive: invalid examples/documentation #5687
• Arrow FLight SQL: invalid location in get_flight_info_prepared_statement #5669 [arrow] [arrow-flight]
• Rust Interval definition incorrect #5654 [parquet] [arrow]
• DECIMAL regex in csv reader does not accept positive exponent specifier #5648 [arrow]
• panic when casting ListArray to FixedSizeList #5642 [arrow]
• FixedSizeListArray::try_new Errors on Entirely Null Array With Size 0 #5614 [arrow]
• parquet / Build wasm32 (pull_request) CI check failing on main #5565 [parquet] [arrow]
• Documentation fix: example in parquet/src/column/mod.rs is incorrect #5560 [parquet]
• IPC code writes data with insufficient alignment #5553 [arrow] [arrow-flight]
• Cannot access example Flight SQL Server from dbeaver #5540 [arrow] [arrow-flight]
• parquet: "not yet implemented" error when codec is actually implemented but disabled #5520 [parquet]

Documentation updates:

• Minor: Improve arrow_cast documentation #5825 [arrow] (alamb)
• Minor: Improve ArrowReaderBuilder::with_row_selection docs #5824 [parquet] (alamb)
• Minor: Add examples for ColumnPath::from #5813 [parquet] (alamb)
• Minor: Clarify docs on EnabledStatistics #5812 [parquet] (alamb)
• Add parquet-derive to repository README #5795 (konjac)
• Refine ParquetRecordBatchReaderBuilder docs #5774 [parquet] (alamb)
• docs: add sizing explanation to bloom filter docs in parquet #5705 [parquet] (hiltontj)

Closed issues:

• binary_mut kernel requires both args to be the same type which is inconsistent with `binary` #5818 [arrow]
• Panic when displaying debug the results via log::info in the browser. #5599 [arrow]

Merged pull requests:

• feat: impl *Assign ops for types in arrow-buffer #5832 [arrow] (waynexia)
• Relax zstd-sys Version Pin #5829 [parquet] (waynexia)
• Minor: Document timestamp with/without cast behavior #5826 [arrow] (alamb)
• fix: invalid examples/documentation in parquet_derive doc #5823 (Weijun-H)
• Check length of FIXED_LEN_BYTE_ARRAY for uuid logical parquet type #5821 [parquet] (mbrobbel)
• Allow overriding the inferred parquet schema root #5814 [parquet] (tustvold)
• Revisit List Row Encoding \#5807 #5811 [arrow] (tustvold)
• Update proc-macro2 requirement from =1.0.83 to =1.0.84 #5805 [arrow] [arrow-flight] (dependabot[bot])
• Fix typo continuation maker -> marker #5802 [arrow] (djanderson)
• fix: serialization of decimal #5801 [arrow] (yjshen)
• Allow constructing ByteViewArray from existing blocks #5796 [arrow] (tustvold)
• Push SortOptions into DynComparator Allowing Nested Comparisons \#5426 #5792 [arrow] (tustvold)
• Fix incorrect URL to Parquet CPP types.h #5790 [parquet] (viirya)
• Update proc-macro2 requirement from =1.0.82 to =1.0.83 #5789 [arrow] [arrow-flight] (dependabot[bot])
• Update prost-build requirement from =0.12.4 to =0.12.6 #5788 [arrow] [arrow-flight] (dependabot[bot])
• Refine parquet documentation on types and metadata #5786 [parquet] (alamb)
• feat(arrow-json): encode Binary and LargeBinary types as hex when writing JSON #5785 [arrow] (hiltontj)
• fix broken link to ballista crate in README.md #5784 (navicore)
• feat(arrow-csv): support encoding of binary in CSV writer #5782 [arrow] (hiltontj)
• Fix documentation for parquet parse_metadata, decode_metadata and decode_footer #5781 [parquet] (alamb)
• Support casting a FixedSizedList<T>[1] to T #5779 [arrow] (sadboy)

• parquet

• Remove harmful table lookup optimization for bitmap operations #5772 [arrow] (HadrienG2)
• Remove deprecated comparison kernels \#4733 #5768 [arrow] (tustvold)
• Add environment variable definitions to run the nanoarrow integration tests #5764 (paleolimbot)
• Downgrade to Rust 1.77 in integration pipeline to fix CI \#5719 #5761 (tustvold)
• Expose boolean builder contents #5760 [arrow] (HadrienG2)
• Allow specifying comment character for CSV reader #5759 [arrow] (bbannier)
• Expose the null buffer of every builder that has one #5754 [arrow] (HadrienG2)
• feat: Make AsyncArrowWriter accepts AsyncFileWriter #5753 [parquet] (Xuanwo)
• Improve repository readme #5752 (alamb)
• Document object store release cadence #5750 (alamb)
• Compute data buffer length by using start and end values in offset buffer #5741 [arrow] (viirya)
• fix: parse string of scientific notation to decimal when the scale is 0 #5740 [arrow] (yjshen)
• Minor: avoid likely unreachable panic in FlightClient #5734 [arrow] [arrow-flight] (alamb)
• Update proc-macro2 requirement from =1.0.81 to =1.0.82 #5732 [arrow] [arrow-flight] (dependabot[bot])
• Improve error message for timestamp queries outside supported range #5730 [arrow] (Abdi-29)
• Refactor to share code between do_put and do_exchange calls #5728 [arrow] [arrow-flight] (opensourcegeek)
• Update brotli requirement from 5.0 to 6.0 #5726 [parquet] (dependabot[bot])
• Fix GenericListBuilder test typo #5724 [arrow] (Kikkon)
• Deprecate NullBuilder capacity, as it behaves in a surprising way #5721 [arrow] (HadrienG2)
• Fix nested nullability when randomly generating arrays #5713 [arrow] (alexwilcoxson-rel)
• Fix up clippy for Rust 1.78 #5710 [parquet] [arrow] (alamb)
• Support casting StringView/BinaryView --> StringArray/BinaryArray. #5704 [arrow] (RinChanNOWWW)
• Fix documentation around handling of nulls in cmp kernels #5697 [arrow] (Jefffrey)
• Support casting StringArray/BinaryArray --> StringView / BinaryView #5686 [arrow] (RinChanNOWWW)
• Add support for flexible column lengths #5679 [arrow] (Posnet)
• Move ffi stream and utils from arrow to arrow-array #5670 [arrow] (alexandreyc)
• Arrow Flight SQL example JDBC driver incompatibility #5666 [arrow] [arrow-flight] (istvan-fodor)
• Add ListView & LargeListView basic construction and validation #5664 [arrow] (Kikkon)
• Update proc-macro2 requirement from =1.0.80 to =1.0.81 #5659 [arrow] [arrow-flight] (dependabot[bot])
• Modify decimal regex to accept positive exponent specifier #5649 [arrow] (jdcasale)
• feat: JSON encoding of FixedSizeList #5646 [arrow] (hiltontj)
• Update proc-macro2 requirement from =1.0.79 to =1.0.80 #5644 [arrow] [arrow-flight] (dependabot[bot])
• fix: panic when casting ListArray to FixedSizeList #5643 [arrow] (jonahgao)
• Add more invalid utf8 parquet reader tests #5639 [parquet] (alamb)
• Update brotli requirement from 4.0 to 5.0 #5637 [parquet] (dependabot[bot])
• Update flatbuffers requirement from 23.1.21 to 24.3.25 #5636 [arrow] (dependabot[bot])
• Increase BinaryViewArray test coverage #5635 [arrow] (alamb)
• PrettyPrint support for StringViewArray and BinaryViewArray #5634 [arrow] (alamb)
• feat(ffi): add run end encoded arrays #5632 [arrow] (notfilippo)
• Accept parquet schemas without explicitly required Map keys #5630 [parquet] (jupiter)
• Implement filter kernel for byte view arrays. #5624 [arrow] (RinChanNOWWW)
• feat: encode FixedSizeBinary in JSON as hex string #5622 [arrow] (hiltontj)
• Update Flight crate README version #5621 [arrow] [arrow-flight] (phillipleblanc)
• feat: support reading and writingStringView and BinaryView in parquet part 1 #5618 [parquet] [arrow] (alamb)
• Use FixedSizeListArray::new in FixedSizeListBuilder #5612 [arrow] (tustvold)
• String to decimal conversion written using E/scientific notation #5611 [arrow] (Nekit2217)
• Account for Timezone when Casting Timestamp to Date32 #5605 [arrow] (Lordworms)
• Update prost-build requirement from =0.12.3 to =0.12.4 #5604 [arrow] [arrow-flight] (dependabot[bot])
• Fix panic when displaying dates on 32-bit platforms #5603 [arrow] (ivanceras)
• Implement take kernel for byte view array. #5602 [arrow] (RinChanNOWWW)
• Add tests for Arrow Flight support for StringViewArray and BinaryViewArray #5601 [arrow] [arrow-flight] (XiangpengHao)
• test: Add a test for RowFilter with nested type #5600 [parquet] (viirya)
• Minor: Add docs for GenericBinaryBuilder, links to GenericStringBuilder #5597 [arrow] (alamb)
• Bump chrono-tz from 0.8 to 0.9 #5596 [arrow] (Jefffrey)
• Update brotli requirement from 3.3 to 4.0 #5586 [parquet] (dependabot[bot])
• Add UnionArray::into_parts #5585 [arrow] (mbrobbel)
• Expose ArrowReaderMetadata::try_new #5583 [parquet] (kylebarron)
• Add try_filter_leaves to propagate error from filter closure #5575 [arrow] (viirya)
• filter for run end array #5573 [arrow] (fabianmurariu)
• Pin zstd-sys to v2.0.9 in parquet #5567 [parquet] (Jefffrey)
• Split arrow_cast::cast::string into it's own submodule #5563 [arrow] (monkwire)
• Correct example code for column \#5560 #5561 [parquet] (zgershkoff)
• Split arrow_cast::cast::dictionary into it's own submodule #5555 [arrow] (monkwire)
• Split arrow_cast::cast::decimal into it's own submodule #5552 [arrow] (monkwire)
• Fix new clippy lints for Rust 1.77 #5544 [parquet] [arrow] (alamb)
• fix: correctly encode ticket #5543 [arrow] [arrow-flight] (freddieptf)
• feat: implemented with_field() for FixedSizeListBuilder #5541 [arrow] (istvan-fodor)
• Split arrow_cast::cast::list into it's own submodule #5537 [arrow] (monkwire)
• Bump black from 22.10.0 to 24.3.0 in /parquet/pytest #5535 [parquet] (dependabot[bot])
• Add OffsetBufferBuilder #5532 [arrow] (tustvold)
• Add IPC StreamDecoder #5531 [arrow] (tustvold)
• IPC format support for StringViewArray and BinaryViewArray #5525 [arrow] (XiangpengHao)
• parquet: Use specific error variant when codec is disabled #5521 [parquet] (progval)
• impl From<ScalarBuffer<T>> for Vec<T> #5518 [arrow] (mbrobbel)

* This Changelog was automatically generated by github_changelog_generator