Граф коммитов

111 Коммитов

Автор SHA1 Сообщение Дата
hsbt 5c1941a9be Merge csv-1.0.2 from upstream.
git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@63364 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
2018-05-09 04:39:16 +00:00
watson1978 dce4a3f58c Improve CSV performance
If it will not use special variables (like $1, $&, $`...),
it can improve the performance by using Regexp#match? or String#match? instead of Regexp#=~ or String#=~.

This patch is same idea as https://github.com/ruby/ruby/pull/1836

[Fix GH-1842]

## Environment
* OS : Ubuntu 17.10
* Compiler : gcc version 7.2.0
* CPU : Intel(R) Core(TM) i5-3210M CPU @ 2.50GHz
* Memory : 16 GB

## TL;DR
Methods     | Before | After  | Speed up
----------- | ------ | ------ | --------
CSV.foreach | 44.825 | 48.201 | 7.5%
CSV#shift   | 45.200 | 49.584 | 9.7%
CSV.read    | 42.968 | 46.853 | 9.0%
CSV.table   | 10.933 | 11.277 | 3.1%

## Before
```
Calculating -------------------------------------
         CSV.foreach     44.825  (± 0.0%) i/s -    228.000  in   5.086576s
           CSV#shift     45.200  (± 0.0%) i/s -    228.000  in   5.044297s
            CSV.read     42.968  (± 0.0%) i/s -    216.000  in   5.027504s
           CSV.table     10.933  (± 0.0%) i/s -     55.000  in   5.031098s
```

## After
```
Calculating -------------------------------------
         CSV.foreach     48.201  (± 0.0%) i/s -    244.000  in   5.062256s
           CSV#shift     49.584  (± 0.0%) i/s -    248.000  in   5.001652s
            CSV.read     46.853  (± 0.0%) i/s -    236.000  in   5.037044s
           CSV.table     11.277  (± 0.0%) i/s -     57.000  in   5.054694s
```

## Benchmark code
```ruby
require 'csv'
require 'benchmark/ips'

CSV.open("/tmp/file.csv", "w") do |csv|
  csv << ["player", "gameA", "gameB"]
  1000.times do
    csv << ['"Alice"', "84.0", "79.5"]
    csv << ['"Bob"', "20.0", "56.5"]
  end
end

Benchmark.ips do |x|
  x.report "CSV.foreach" do
    CSV.foreach("/tmp/file.csv") do |row|
    end
  end

  x.report "CSV#shift" do
    CSV.open("/tmp/file.csv") do |csv|
      while line = csv.shift
      end
    end
  end

  x.report "CSV.read" do
    CSV.read("/tmp/file.csv")
  end

  x.report "CSV.table" do
    CSV.table("/tmp/file.csv")
  end
end
```

git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@62806 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
2018-03-18 10:28:58 +00:00
hsbt 0593c6a510 Merge csv-0.1.0 from upstream.
* csv.gemspec: Update release version.
  * lib/csv.rb: Remove superfluous private clause.

git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@59849 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
2017-09-12 08:38:06 +00:00
hsbt 9d99093169 Partly reverted r59642. Because IO#close is idempotent since Ruby 2.3.
Reported by Eric Wong. Thank you.

git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@59683 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
2017-08-29 10:22:47 +00:00
glass 1a1930180c lib/csv.rb: refactor and optimize. This change
includes the patch from marshall-lee.
close #1168

git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@59671 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
2017-08-28 07:54:00 +00:00
glass 20c5f60775 csv.rb: optimize CSV::Table#to_a and #to_csv
* lib/csv.rb (CSV::Table#to_a, #to_csv): use Array#push instead of
  Array#concat for performance improvement. This performance improvement is
  proposed by zdennis <zach.dennis@gmail.com>. The patch is from
  Mau Magnaguagno <maumagnaguagno@gmail.com>.
  close #946

git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@59657 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
2017-08-25 08:13:32 +00:00
hsbt ffb49a7e9b Fixed regression to convert blank value at r45497.
[Bug #11126][ruby-core:69088]

git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@59643 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
2017-08-22 06:57:00 +00:00
hsbt ede0df3a9b Gracefully handle CSV IO when file descriptor closed.
[Bug #10504][ruby-core:66240]

git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@59642 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
2017-08-22 06:40:59 +00:00
hsbt 20aed4495b Fixed equality method fails when given the object that doesn't support table method.
[Bug #12422][ruby-core:75707]

git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@59641 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
2017-08-22 06:34:27 +00:00
hsbt 9c38a08f28 Fixed to write_headers option behavior when given no rows.
[Bug #9988][ruby-core:63375]

git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@59640 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
2017-08-22 06:17:42 +00:00
hsbt 2c69f7b278 Escape skip_lines string before convert to Regexp.
It ignored all of lines when given Regexp special characters.

  [Feature #9147][ruby-core:58549]

git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@59639 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
2017-08-22 05:32:34 +00:00
glass 32eeff1708 csv.rb: fix incompatibility introduced in r59428
* lib/csv.rb: fix incompatibility introduced in r59428.
              CSV.new takes options as keyword arguments.

* test/csv/test_features.rb: add a test to ensure it raises error againt
                             unknown options

* test/csv/test_features.rb: add a test to ensure row_sep option is properly
                             applied

git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@59437 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
2017-07-28 07:46:20 +00:00
glass a983fc5022 csv.rb: use keyword parameters
* lib/csv.rb: usb keyword parameters to receive options

* test/csv/test_features.rb: remove a test for checking options

git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@59428 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
2017-07-27 09:53:58 +00:00
glass 5e3f1a10ca csv.rb: use Array#to_h
* lib/csv.rb (CSV::Row#to_hash): use Array#to_h

git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@59426 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
2017-07-27 06:49:27 +00:00
kazu 3f8248f1ee Remove redundant square brackets
Use character class directly instead of
character class in character class.

git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@58827 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
2017-05-21 01:01:10 +00:00
hsbt 37abc2fb7e Improve CSV parsing performance.
Patch by @joshpencheon (Josh Pencheon)
  [fix GH-1607]

  #### benchmark-ips results
  ```
  trunk:
  Warming up --------------------------------------
                         4.000  i/100ms
  Calculating -------------------------------------
                         39.661  (±10.1%) i/s -      2.352k in 60.034781s
  with-patch:
  Warming up --------------------------------------
                         5.000  i/100ms
  Calculating -------------------------------------
                         60.521  (± 9.9%) i/s -      3.595k in 60.047157s
  ```

  #### memory_profiler resuts

  ```
  trunk:
  allocated memory by class
  -----------------------------------
    35588490  String
     7454320  Array
      294000  MatchData
       37340  Regexp
       11840  Hash
        2400  CSV
        1600  Proc
        1280  Method
         800  StringIO
  with-patch:
  allocated memory by class
  -----------------------------------
    18788490  String
     3454320  Array
      294000  MatchData
       37340  Regexp
       11840  Hash
        2400  CSV
        1600  Proc
        1280  Method
         800  StringIO
  ```

git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@58777 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
2017-05-18 02:42:16 +00:00
hsbt 15cb9817a2 Optimize CSV#shift.
[Bug #12373][ruby-core:75462]
  Patch by Yuki Kurihara.

  Benchmark:
  ```
  Warming up --------------------------------------
           csv_shift     1.000  i/100ms
       new_csv_shift     1.000  i/100ms
Calculating -------------------------------------
           csv_shift      1.192  (± 0.0%) i/s -      6.000  in 5.034250s
       new_csv_shift      1.527  (± 0.0%) i/s -      8.000  in 5.243446s

Comparison:
       new_csv_shift:        1.5 i/s
           csv_shift:        1.2 i/s - 1.28x  slower
  ```

git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@58770 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
2017-05-17 07:56:27 +00:00
hsbt 87acdae43d Strip punctuation from CSV headers in symbol converter.
Patch by @cllns. [Fix GH-957]

git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@58744 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
2017-05-16 09:32:32 +00:00
hsbt 3b77cb2a64 Added accessor of original line when parsing.
[Feature #11865][ruby-core:72452][fix GH-1170]

git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@58743 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
2017-05-16 09:17:09 +00:00
nobu 5bb125c63e csv.rb: fix field_size_limit check
* lib/csv.rb (CSV#shift): the last column is an Array in extended
  column since r55985.  [ruby-dev:49964] [Bug #13149]

git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@57406 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
2017-01-23 05:59:28 +00:00
ktsj b425d4f19a * lib/csv.rb (CSV::{Row,Table}#{each,delete_if}): returns an enumerator
if no block is given. [ruby-core:75346] [Feature #11058]

* test/csv/test_row.rb: add test for above.

* test/csv/test_table.rb: ditto.

git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@56083 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
2016-09-07 06:06:09 +00:00
nobu 4e5114b0d1 csv.rb: performance with very long quoted lines
* lib/csv.rb (CSV#shift): store partial quoted strings in an array
  and join at last, to improve performance with very long quoted
  lines.  [ruby-core:76987] [Bug #12691]

git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@55985 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
2016-08-22 07:29:54 +00:00
nobu ba5a40fe7d Update documentation of CSV header converter
* lib/csv.rb: Update documentation of CSV header converter for
  r45498, [GH-575].  [Fix GH-1215]

git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@53628 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
2016-01-23 02:30:07 +00:00
jeg2 f18f940802 Adding a liberal_parsing option to CSV. Patch by Braden Anderson.
git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@53401 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
2016-01-01 02:44:48 +00:00
nobu 0d1a243af1 csv.rb: Fix typo [ci skip
* lib/csv.rb (CSV#initialize): [DOC] Fix double-word typo.
  [Fix GH-1161]

git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@53199 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
2015-12-19 05:23:50 +00:00
nobu a6e055f4a1 csv.rb: tail commas
* lib/csv.rb (CSV::Converters, CSV::DEFAULT_OPTIONS): supply tail
  commas for further elements.

git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@53184 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
2015-12-18 03:55:29 +00:00
nobu 3bf1c09c7a csv.rb: fix typo
* lib/csv.rb (CSV#shift): fix typo.  [See GH-1160]

git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@53183 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
2015-12-18 03:55:08 +00:00
nobu b14f420995 csv.rb: fix encoding
* lib/csv.rb: encoding must be plased at the first line except for
  shebang.  [fix GH-1116]

git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@52850 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
2015-12-02 02:40:02 +00:00
hsbt aed0db8bd3 * lib/csv.rb: enable frozen_string_literal.
[fix GH-1116] Patch by @marshall-lee

git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@52848 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
2015-12-02 02:23:12 +00:00
hsbt 601c10e6ec * lib/csv.rb: typo fix [ci skip][fix GH-958] Patch by @henrik
git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@51191 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
2015-07-07 22:43:11 +00:00
hsbt 96fb819676 * lib/csv.rb: accept to use Range object for row selection.
[Feature #11267][ruby-dev:49091]

git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@50929 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
2015-06-17 01:58:34 +00:00
gogotanaka 1781c4c1e1 * Fix typo. Patch by @davydovanton
[Fix GH-928][ci skip]

git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@50813 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
2015-06-10 05:06:25 +00:00
gogotanaka 2351d88f51 lib/csv.rb: [DOC] Fix typo [ci skip][Fix GH-907]
git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@50709 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
2015-06-01 05:35:42 +00:00
svn ab337e61ec * remove trailing spaces.
git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@48259 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
2014-11-04 03:21:53 +00:00
hsbt 934789941e * lib/csv.rb: added documentation for skip_blanks option by @decasia
[fix GH-744][ci skip]

git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@48258 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
2014-11-04 03:21:50 +00:00
glass 5be5db6350 * lib/csv.rb: avoid unnecessary object allocations.
patch is from Andrew Vit. [ruby-core:63215] [Feature #9952]

git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@47663 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
2014-09-21 03:21:17 +00:00
nobu f9a5335ed4 csv.rb: honor encoding option
* lib/csv.rb (CSV#<<): honor explicity given encoding.  based on
  the patch by DAISUKE TANIWAKI <daisuketaniwaki AT gmail.com> at
  [ruby-core:62113].  [Bug #9766]

git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@46391 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
2014-06-10 01:57:10 +00:00
akr 2ad2007307 * lib/csv.rb (CSV.open): Close the opened file when an exception occur.
git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@46224 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
2014-05-29 10:44:59 +00:00
zzak 0aa197c6f2 * lib/csv.rb: Reject nil as data source for CSV.new, patch by @Peeja.
[Fixes GH-580] https://github.com/ruby/ruby/pull/580


git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@46123 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
2014-05-25 16:41:40 +00:00
hsbt 22305ffea0 * lib/csv.rb: fix minor typo by @baroquebobcat [fix GH-583][ci skip]
git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@45510 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
2014-04-03 05:59:14 +00:00
jeg2 1170b057e0 * lib/csv.rb: Symbol HeaderConverter: strip leading/trailing space.
Reported by Skye Shaw
  [Fixes GH-575]



git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@45498 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
2014-04-01 14:40:48 +00:00
jeg2 7c4d4e1eb2 * lib/csv.rb: Don't attempt to convert nil headers.
Reported by Skye Shaw



git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@45497 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
2014-04-01 14:35:03 +00:00
jeg2 3fb752a921 * lib/csv.rb: Fixed a broken regular expression that was causing
CSV to miss escaping some special meaning characters when used
  in parsing.
  Reported by David Unric
  [ruby-core:54986] [Bug #8405]



git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@45374 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
2014-03-20 14:38:21 +00:00
a_matsuda 01b1f00461 * lib/csv.rb: [DOC] Fix typos
s/occurrs/occurs/, s/interation/iteration/, s/intetionally/intentionally/,
s/modfied/modified/, s/axception/exception/, s/curcuit/circuit/
[ci skip]

git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@43955 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
2013-12-02 05:36:00 +00:00
jeg2 895e9b0acd * lib/csv.rb: Optimize header hashes by freezing string keys.
[ruby-core:58510]



git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@43825 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
2013-11-24 00:44:41 +00:00
jeg2 80c4b4b3d7 * lib/csv.rb: If skip_lines is set to a String, convert it to a Regexp
to prevent the alternative, which is that each line in the CSV gets
  converted to a Regexp when calling skip_lines#match.



git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@43823 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
2013-11-23 23:12:11 +00:00
zzak bceb856986 * lib/csv.rb: [DOC] Fix typo in CSV.parse_line by @funky-bibimbap
[Fixes GH-430] https://github.com/ruby/ruby/pull/430


git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@43569 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
2013-11-07 17:10:34 +00:00
nobu 28204c67ed csv.rb: foreach enumerator
* lib/csv.rb (CSV.foreach): support enumerator.  based on a patch by
  Hanmac (Hans Mackowiak) at [ruby-core:57643].  [ruby-core:57283]
  [Feature #8929]

git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@43135 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
2013-10-04 00:32:35 +00:00
nobu 45c2532e59 csv.rb: get rid of discarding coderange
* lib/csv.rb (CSV#<<): use StringIO#set_encoding instead of creating
  new StringIO instance with String#force_encoding, forcing encoding
  discards the cached coderange bits and can make further operations
  very slow.  [ruby-core:55714] [Bug #8585]

git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@41722 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
2013-07-01 06:05:03 +00:00
jeg2 b614d7823c * lib/csv.rb: Remove the dangerous serialization feature.
git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@39077 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
2013-02-05 18:28:26 +00:00