github
/
ruby
зеркало из https://github.com/github/ruby.git
1
0
Форкнуть 0
Код Задачи Пакеты Проекты Релизы Вики Активность

* lib/csv.rb: writes lines with "\n" when row separator is not given. 

formerly it was "\r\n".

        * lib/csv.rb: [CAUTION] API change

          * CSV::Row removed.  a row is represented as just an Array.  since
            CSV::Row was a subclass of Array, it won't hurt almost all programs
            except one which depended CSV::Row#match.

          * CSV::Cell removed.  a cell is represented as just a String or
            nil(NULL).  this change will cause widespread destruction.

              CSV.open("foo.csv", "r") do |row|
                row.each do |cell|
                  if cell.is_null       # Cell#is_null
                    p "(NULL)"
                  else
                    p cell.data         # Cell#data
                  end
                end
              end

            must be just;

              CSV.open("foo.csv", "r") do |row|
                row.each do |cell|
                  if cell.nil?
                    p "(NULL)"
                  else
                    p cell
                  end
                end
              end

        * lib/csv.rb: [CAUTION] record separator(CR, LF, CR+LF) behavior
          change.  CSV.open, CSV.parse, and CSV,generate now do not force
          opened file binmode.  formerly it set binmode explicitly.

          with CSV.open, binmode of opened file depends the given mode
          parameter "r", "w", "rb", and "wb".  CSV.parse and CSV.generate open
          file with "r" and "w".

          setting mode properly is user's responsibility now.

        * lib/csv.rb: accepts String as a fs (field separator/column separator)
          and rs (record separator/row separator)

        * lib/csv.rb: added CSV.foreach(path, rs = nil, &block).  CSV.foreach
          now does not handle "| cmd" as a path different from IO.foreach.
          needed?

        * test/csv/test_csv.rb: updated.


git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@6359 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
This commit is contained in:
nahi 2004-05-18 12:34:33 +00:00
Родитель 75db654808
Коммит 34a553da2e
3 изменённых файлов: 471 добавлений и 619 удалений
Показать статистику Скачать Patch файл Скачать Diff файл Показать все файлы Свернуть все файлы

55
ChangeLog
Просмотреть файл

@ -1,3 +1,58 @@
Tue May 18 21:21:43 2004 NAKAMURA, Hiroshi <nakahiro@sarion.co.jp>
* lib/csv.rb: writes lines with "\n" when row separator is not given.
formerly it was "\r\n".
* lib/csv.rb: [CAUTION] API change
* CSV::Row removed. a row is represented as just an Array. since
CSV::Row was a subclass of Array, it won't hurt almost all programs
except one which depended CSV::Row#match.
* CSV::Cell removed. a cell is represented as just a String or
nil(NULL). this change will cause widespread destruction.
CSV.open("foo.csv", "r") do |row|
row.each do |cell|
if cell.is_null # Cell#is_null
p "(NULL)"
else
p cell.data # Cell#data
end
end
end
must be just;
CSV.open("foo.csv", "r") do |row|
row.each do |cell|
if cell.nil?
p "(NULL)"
else
p cell
end
end
end
* lib/csv.rb: [CAUTION] record separator(CR, LF, CR+LF) behavior
change. CSV.open, CSV.parse, and CSV,generate now do not force
opened file binmode. formerly it set binmode explicitly.
with CSV.open, binmode of opened file depends the given mode
parameter "r", "w", "rb", and "wb". CSV.parse and CSV.generate open
file with "r" and "w".
setting mode properly is user's responsibility now.
* lib/csv.rb: accepts String as a fs (field separator/column separator)
and rs (record separator/row separator)
* lib/csv.rb: added CSV.foreach(path, rs = nil, &block). CSV.foreach
now does not handle "| cmd" as a path different from IO.foreach.
needed?
* test/csv/test_csv.rb: updated.
Tue May 18 14:24:20 2004 why the lucky stiff <why@ruby-lang.org>
* lib/yaml.rb: added rdoc to beginning of lib.

549
lib/csv.rb
Просмотреть файл

@ -1,106 +1,30 @@
# CSV -- module for generating/parsing CSV data.
# Copyright (C) 2000-2004 NAKAMURA, Hiroshi <nakahiro@sarion.co.jp>.
# $Id$
# This program is copyrighted free software by NAKAMURA, Hiroshi. You can
# redistribute it and/or modify it under the same terms of Ruby's license;
# either the dual license version in 2003, or any later version.
class CSV
# Describes a cell of CSV.
class Cell
# Datum as string.
attr_accessor :data
# Is this datum NULL?
attr_accessor :is_null
# If is_null is true, datum is stored in the instance created but it
# should be treated as 'NULL'.
def initialize(data = '', is_null = true)
@data = data
@is_null = is_null
end
# Compares another cell with self. Bear in mind NULL matches with NULL.
# Use CSV::Cell#== if you don't want NULL matches with NULL.
# rhs: an instance of CSV::Cell to be compared.
def match(rhs)
if @is_null and rhs.is_null
true
elsif @is_null or rhs.is_null
false
else
@data == rhs.data
end
end
# Compares another cell with self. Bear in mind NULL does not match with
# NULL. Use CSV::Cell#match if you want NULL matches with NULL.
# rhs: an instance of CSV::Cell to be compared.
def ==(rhs)
if @is_null or rhs.is_null
false
else
@data == rhs.data
end
end
def to_str
content.to_str
end
def to_s
content.to_s
end
private
def content
@is_null ? nil : data
end
end
# Describes a row of CSV. Each element must be a CSV::Cell.
class Row < Array
# Returns the strings contained in the row's cells.
def to_a
self.collect { |cell| cell.is_null ? nil : cell.data }
end
# Compares another row with self.
# rhs: an Array of cells. Each cell should be a CSV::Cell.
def match(rhs)
if self.size != rhs.size
return false
end
for idx in 0...(self.size)
unless self[idx].match(rhs[idx])
return false
end
end
true
end
end
class IllegalFormatError < RuntimeError; end
def CSV.open(filename, mode, col_sep = ?,, row_sep = nil, &block)
def CSV.open(path, mode, fs = ',', rs = nil, &block)
if mode == 'r' or mode == 'rb'
open_reader(filename, col_sep, row_sep, &block)
open_reader(path, mode, fs, rs, &block)
elsif mode == 'w' or mode == 'wb'
open_writer(filename, col_sep, row_sep, &block)
open_writer(path, mode, fs, rs, &block)
else
raise ArgumentError.new("'mode' must be 'r', 'rb', 'w', or 'wb'")
end
end
def CSV.foreach(path, rs = nil, &block)
open_reader(path, 'r', ',', rs, &block)
end
# Open a CSV formatted file for reading.
#
# EXAMPLE 1
@ -127,8 +51,8 @@ class CSV
# RETURNS
# reader instance. To get parse result, see CSV::Reader#each.
#
def CSV.parse(filename, col_sep = ?,, row_sep = nil, &block)
open_reader(filename, col_sep, row_sep, &block)
def CSV.parse(path, fs = ',', rs = nil, &block)
open_reader(path, 'r', fs, rs, &block)
end
# Open a CSV formatted file for writing.
@ -156,8 +80,8 @@ class CSV
# writer instance. See CSV::Writer#<< and CSV::Writer#add_row to know how
# to generate CSV string.
#
def CSV.generate(filename, col_sep = ?,, row_sep = nil, &block)
open_writer(filename, col_sep, row_sep, &block)
def CSV.generate(path, fs = ',', rs = nil, &block)
open_writer(path, 'w', fs, rs, &block)
end
# Parse a line from given string. Bear in mind it parses ONE LINE. Rest of
@ -166,47 +90,52 @@ class CSV
#
# If you don't know whether a target string to parse is exactly 1 line or
# not, use CSV.parse_row instead of this method.
def CSV.parse_line(src, col_sep = ?,, row_sep = nil)
def CSV.parse_line(src, fs = ',', rs = nil)
if !fs.nil? and fs.is_a?(Fixnum)
fs = fs.chr
end
if !rs.nil? and rs.is_a?(Fixnum)
rs = rs.chr
end
idx = 0
res_type = :DT_COLSEP
cells = Row.new
row = []
begin
while (res_type.equal?(:DT_COLSEP))
cell = Cell.new
res_type, idx = parse_body(src, idx, cell, col_sep, row_sep)
cells.push(cell.is_null ? nil : cell.data)
res_type, idx, cell = parse_body(src, idx, fs, rs)
row << cell
end
rescue IllegalFormatError
return Row.new
return []
end
cells
row
end
# Create a line from cells. each cell is stringified by to_s.
def CSV.generate_line(cells, col_sep = ?,, row_sep = nil)
if (cells.size == 0)
def CSV.generate_line(row, fs = ',', rs = nil)
if (row.size == 0)
return ''
end
if !fs.nil? and fs.is_a?(Fixnum)
fs = fs.chr
end
if !rs.nil? and rs.is_a?(Fixnum)
rs = rs.chr
end
res_type = :DT_COLSEP
result_str = ''
idx = 0
while true
cell = if (cells[idx].nil?)
Cell.new('', true)
else
Cell.new(cells[idx].to_s, false)
end
generate_body(cell, result_str, col_sep, row_sep)
generate_body(row[idx], result_str, fs, rs)
idx += 1
if (idx == cells.size)
if (idx == row.size)
break
end
generate_separator(:DT_COLSEP, result_str, col_sep, row_sep)
generate_separator(:DT_COLSEP, result_str, fs, rs)
end
result_str
end
# Parse a line from string. Consider using CSV.parse_line instead.
# To parse lines in CSV string, see EXAMPLE below.
#
@ -236,16 +165,21 @@ class CSV
# parsed_cells: num of parsed cells.
# idx: index of next parsing location of 'src'.
#
def CSV.parse_row(src, idx, out_dev, col_sep = ?,, row_sep = nil)
def CSV.parse_row(src, idx, out_dev, fs = ',', rs = nil)
if !fs.nil? and fs.is_a?(Fixnum)
fs = fs.chr
end
if !rs.nil? and rs.is_a?(Fixnum)
rs = rs.chr
end
idx_backup = idx
parsed_cells = 0
res_type = :DT_COLSEP
begin
while (!res_type.equal?(:DT_ROWSEP))
cell = Cell.new
res_type, idx = parse_body(src, idx, cell, col_sep, row_sep)
res_type, idx, cell = parse_body(src, idx, fs, rs)
if res_type.equal?(:DT_EOS)
if idx == idx_backup #((parsed_cells == 0) && (cell.is_null))
if idx == idx_backup #((parsed_cells == 0) and cell.nil?)
return 0, 0
end
res_type = :DT_ROWSEP
@ -258,8 +192,7 @@ class CSV
end
return parsed_cells, idx
end
# Convert a line from cells data to string. Consider using CSV.generate_line
# instead. To generate multi-row CSV string, see EXAMPLE below.
#
@ -292,39 +225,46 @@ class CSV
# RETURNS
# parsed_cells: num of converted cells.
#
def CSV.generate_row(src, cells, out_dev, col_sep = ?,, row_sep = nil)
def CSV.generate_row(src, cells, out_dev, fs = ',', rs = nil)
if !fs.nil? and fs.is_a?(Fixnum)
fs = fs.chr
end
if !rs.nil? and rs.is_a?(Fixnum)
rs = rs.chr
end
src_size = src.size
if (src_size == 0)
if cells == 0
generate_separator(:DT_ROWSEP, out_dev, col_sep, row_sep)
generate_separator(:DT_ROWSEP, out_dev, fs, rs)
end
return 0
end
res_type = :DT_COLSEP
parsed_cells = 0
generate_body(src[parsed_cells], out_dev, col_sep, row_sep)
generate_body(src[parsed_cells], out_dev, fs, rs)
parsed_cells += 1
while ((parsed_cells < cells) && (parsed_cells != src_size))
generate_separator(:DT_COLSEP, out_dev, col_sep, row_sep)
generate_body(src[parsed_cells], out_dev, col_sep, row_sep)
while ((parsed_cells < cells) and (parsed_cells != src_size))
generate_separator(:DT_COLSEP, out_dev, fs, rs)
generate_body(src[parsed_cells], out_dev, fs, rs)
parsed_cells += 1
end
if (parsed_cells == cells)
generate_separator(:DT_ROWSEP, out_dev, col_sep, row_sep)
generate_separator(:DT_ROWSEP, out_dev, fs, rs)
else
generate_separator(:DT_COLSEP, out_dev, col_sep, row_sep)
generate_separator(:DT_COLSEP, out_dev, fs, rs)
end
parsed_cells
end
# Private class methods.
class << self
private
def open_reader(filename, col_sep, row_sep, &block)
file = File.open(filename, 'rb')
def open_reader(path, mode, fs, rs, &block)
file = File.open(path, mode)
if block
begin
CSV::Reader.parse(file, col_sep, row_sep) do |row|
CSV::Reader.parse(file, fs, rs) do |row|
yield(row)
end
ensure
@ -332,17 +272,17 @@ class CSV
end
nil
else
reader = CSV::Reader.create(file, col_sep, row_sep)
reader = CSV::Reader.create(file, fs, rs)
reader.close_on_terminate
reader
end
end
def open_writer(filename, col_sep, row_sep, &block)
file = File.open(filename, 'wb')
def open_writer(path, mode, fs, rs, &block)
file = File.open(path, mode)
if block
begin
CSV::Writer.generate(file, col_sep, row_sep) do |writer|
CSV::Writer.generate(file, fs, rs) do |writer|
yield(writer)
end
ensure
@ -350,147 +290,175 @@ class CSV
end
nil
else
writer = CSV::Writer.create(file, col_sep, row_sep)
writer = CSV::Writer.create(file, fs, rs)
writer.close_on_terminate
writer
end
end
def parse_body(src, idx, cell, col_sep, row_sep)
row_sep_end = row_sep || ?\n
cell.is_null = false
def parse_body(src, idx, fs, rs)
fs_str = fs
fs_size = fs_str.size
fs_idx = 0
rs_str = rs || "\n"
rs_size = rs_str.size
rs_idx = 0
cell = ''
state = :ST_START
quoted = false
cr = false
c = nil
last_idx = idx
while (c = src[idx])
idx += 1
result_state = :DT_UNKNOWN
if (c == col_sep)
if state.equal?(:ST_DATA)
if cr
raise IllegalFormatError.new
end
if (!quoted)
state = :ST_END
result_state = :DT_COLSEP
else
cell.data << c.chr
end
elsif state.equal?(:ST_QUOTE)
if cr
raise IllegalFormatError.new
end
state = :ST_END
result_state = :DT_COLSEP
else # :ST_START
cell.is_null = true
state = :ST_END
result_state = :DT_COLSEP
if c == ?"
cell << src[last_idx, (idx - last_idx)]
last_idx = idx
if cr
raise IllegalFormatError
end
if fs_idx != 0
fs_idx = 0
end
if rs_idx != 0
rs_idx = 0
end
elsif (c == ?") # " for vim syntax hilighting.
if state.equal?(:ST_DATA)
if cr
raise IllegalFormatError.new
end
if quoted
last_idx += 1
quoted = false
state = :ST_QUOTE
else
raise IllegalFormatError.new
raise IllegalFormatError
end
elsif state.equal?(:ST_QUOTE)
cell.data << c.chr
cell << c.chr
last_idx += 1
quoted = true
state = :ST_DATA
else # :ST_START
quoted = true
last_idx += 1
state = :ST_DATA
end
elsif row_sep.nil? and c == ?\r
if cr
raise IllegalFormatError.new
elsif c == fs_str[fs_idx]
fs_idx += 1
cell << src[last_idx, (idx - last_idx)]
last_idx = idx
if rs_idx != 0
rs_idx = 0
end
if fs_idx == fs_size
fs_idx = 0
if cr
raise IllegalFormatError
end
if state.equal?(:ST_DATA)
if rs_idx != 0
cell << rs_str[0, rs_idx]
rs_idx = 0
end
if quoted
true # ToDo: delete; dummy line for coverage
else
return :DT_COLSEP, idx + 1, cell;
end
elsif state.equal?(:ST_QUOTE)
if rs_idx != 0
raise IllegalFormatError
end
return :DT_COLSEP, idx + 1, cell;
else # :ST_START
return :DT_COLSEP, idx + 1, nil
end
end
elsif c == rs_str[rs_idx]
rs_idx += 1
unless (rs.nil? and cr)
cell << src[last_idx, (idx - last_idx)]
last_idx = idx
end
if fs_idx != 0
fs_idx = 0
end
if rs_idx == rs_size
rs_idx = 0
if state.equal?(:ST_DATA)
if quoted
true # ToDo: delete; dummy line for coverage
else
return :DT_ROWSEP, idx + 1, cell
end
elsif state.equal?(:ST_QUOTE)
return :DT_ROWSEP, idx + 1, cell
else # :ST_START
return :DT_ROWSEP, idx + 1, nil
end
end
elsif rs.nil? and c == ?\r
# special \r treatment for backward compatibility
if cr
raise IllegalFormatError
end
cell << src[last_idx, (idx - last_idx)]
last_idx = idx
if quoted
cell.data << c.chr
state = :ST_DATA
else
cr = true
end
elsif c == row_sep_end
if state.equal?(:ST_DATA)
if cr
state = :ST_END
result_state = :DT_ROWSEP
cr = false
else
if quoted
cell.data << c.chr
state = :ST_DATA
else
state = :ST_END
result_state = :DT_ROWSEP
end
end
elsif state.equal?(:ST_QUOTE)
state = :ST_END
result_state = :DT_ROWSEP
if cr
cr = false
end
else # :ST_START
cell.is_null = true
state = :ST_END
result_state = :DT_ROWSEP
end
else
if state.equal?(:ST_DATA) || state.equal?(:ST_START)
if fs_idx != 0
fs_idx = 0
end
if rs_idx != 0
rs_idx = 0
end
if state.equal?(:ST_DATA) or state.equal?(:ST_START)
if cr
raise IllegalFormatError.new
raise IllegalFormatError
end
cell.data << c.chr
state = :ST_DATA
else # :ST_QUOTE
raise IllegalFormatError.new
raise IllegalFormatError
end
end
if state.equal?(:ST_END)
return result_state, idx;
end
idx += 1
end
if state.equal?(:ST_START)
cell.is_null = true
elsif state.equal?(:ST_QUOTE)
true # dummy for coverate; only a data
return :DT_EOS, idx, nil
elsif quoted
raise IllegalFormatError.new
raise IllegalFormatError
elsif cr
raise IllegalFormatError.new
raise IllegalFormatError
end
return :DT_EOS, idx
cell << src[last_idx, (idx - last_idx)]
last_idx = idx
return :DT_EOS, idx, cell
end
def generate_body(cells, out_dev, col_sep, row_sep)
row_data = cells.data.dup
if (!cells.is_null)
if (row_data.gsub!('"', '""') ||
row_data.include?(col_sep) ||
(row_sep && row_data.index(row_sep)) ||
(/[\r\n]/ =~ row_data) ||
(cells.data.empty?))
def generate_body(cell, out_dev, fs, rs)
if cell.nil?
# empty
else
row_data = cell.dup
if (row_data.gsub!('"', '""') or
row_data.index(fs) or
(rs and row_data.index(rs)) or
(/[\r\n]/ =~ row_data) or
(cell.empty?))
out_dev << '"' << row_data << '"'
else
out_dev << row_data
end
end
end
def generate_separator(type, out_dev, col_sep, row_sep)
def generate_separator(type, out_dev, fs, rs)
case type
when :DT_COLSEP
out_dev << col_sep.chr
out_dev << fs
when :DT_ROWSEP
out_dev << (row_sep ? row_sep.chr : "\r\n")
out_dev << (rs || "\n")
end
end
end
@ -499,7 +467,7 @@ class CSV
# CSV formatted string/stream reader.
#
# EXAMPLE
# read CSV lines until the first column is 'stop'.
# read CSV lines untill the first column is 'stop'.
#
# CSV::Reader.parse(File.open('bigdata', 'rb')) do |row|
# p row
@ -511,8 +479,8 @@ class CSV
# Parse CSV data and get lines. Given block is called for each parsed row.
# Block value is always nil. Rows are not cached for performance reason.
def Reader.parse(str_or_readable, col_sep = ?,, row_sep = nil)
reader = create(str_or_readable, col_sep, row_sep)
def Reader.parse(str_or_readable, fs = ',', rs = nil)
reader = create(str_or_readable, fs, rs)
reader.each do |row|
yield(row)
end
@ -521,20 +489,20 @@ class CSV
end
# Returns reader instance.
def Reader.create(str_or_readable, col_sep = ?,, row_sep = nil)
def Reader.create(str_or_readable, fs = ',', rs = nil)
case str_or_readable
when IO
IOReader.new(str_or_readable, col_sep, row_sep)
IOReader.new(str_or_readable, fs, rs)
when String
StringReader.new(str_or_readable, col_sep, row_sep)
StringReader.new(str_or_readable, fs, rs)
else
IOReader.new(str_or_readable, col_sep, row_sep)
IOReader.new(str_or_readable, fs, rs)
end
end
def each
while true
row = Row.new
row = []
parsed_cells = get_row(row)
if parsed_cells == 0
break
@ -545,7 +513,7 @@ class CSV
end
def shift
row = Row.new
row = []
parsed_cells = get_row(row)
row
end
@ -557,25 +525,23 @@ class CSV
private
def initialize(dev)
raise RuntimeError.new('do not instantiate this class directly')
raise RuntimeError.new('Do not instanciate this class directly.')
end
def get_row(row)
raise NotImplementedError.new(
'method get_row must be defined in a derived class')
raise NotImplementedError.new('Method get_row must be defined in a derived class.')
end
def terminate
# Define if needed.
end
end
class StringReader < Reader
def initialize(string, col_sep = ?,, row_sep = nil)
@col_sep = col_sep
@row_sep = row_sep
def initialize(string, fs = ',', rs = nil)
@fs = fs
@rs = rs
@dev = string
@idx = 0
if @dev[0, 3] == "\xef\xbb\xbf"
@ -586,9 +552,8 @@ class CSV
private
def get_row(row)
parsed_cells, next_idx =
CSV.parse_row(@dev, @idx, row, @col_sep, @row_sep)
if parsed_cells == 0 && next_idx == 0 && @idx != @dev.size
parsed_cells, next_idx = CSV.parse_row(@dev, @idx, row, @fs, @rs)
if parsed_cells == 0 and next_idx == 0 and @idx != @dev.size
raise IllegalFormatError.new
end
@idx = next_idx
@ -598,12 +563,10 @@ class CSV
class IOReader < Reader
def initialize(io, col_sep = ?,, row_sep = nil)
def initialize(io, fs = ',', rs = nil)
@io = io
@io.binmode if @io.respond_to?(:binmode)
@col_sep = col_sep
@row_sep = row_sep
@fs = fs
@rs = rs
@dev = CSV::IOBuf.new(@io)
@idx = 0
if @dev[0] == 0xef and @dev[1] == 0xbb and @dev[2] == 0xbf
@ -621,9 +584,8 @@ class CSV
private
def get_row(row)
parsed_cells, next_idx =
CSV.parse_row(@dev, @idx, row, @col_sep, @row_sep)
if parsed_cells == 0 && next_idx == 0 && !@dev.is_eos?
parsed_cells, next_idx = CSV.parse_row(@dev, @idx, row, @fs, @rs)
if parsed_cells == 0 and next_idx == 0 and !@dev.is_eos?
raise IllegalFormatError.new
end
dropped = @dev.drop(next_idx)
@ -667,40 +629,25 @@ class CSV
# outfile.close
#
class Writer
# Generate CSV. Given block is called with the writer instance.
def Writer.generate(str_or_writable, col_sep = ?,, row_sep = nil)
writer = Writer.create(str_or_writable, col_sep, row_sep)
def Writer.generate(str_or_writable, fs = ',', rs = nil)
writer = Writer.create(str_or_writable, fs, rs)
yield(writer)
writer.close
nil
end
# str_or_writable must handle '<<(string)'.
def Writer.create(str_or_writable, col_sep = ?,, row_sep = nil)
BasicWriter.new(str_or_writable, col_sep, row_sep)
def Writer.create(str_or_writable, fs = ',', rs = nil)
BasicWriter.new(str_or_writable, fs, rs)
end
# dump CSV stream to the device. argument must be an Array of String.
def <<(ary)
row = ary.collect { |item|
if item.is_a?(Cell)
item
elsif (item.nil?)
Cell.new('', true)
else
Cell.new(item.to_s, false)
end
}
CSV.generate_row(row, row.size, @dev, @col_sep, @row_sep)
self
end
# dump CSV stream to the device. argument must be an Array of CSV::Cell.
def add_row(row)
CSV.generate_row(row, row.size, @dev, @col_sep, @row_sep)
def <<(row)
CSV.generate_row(row, row.size, @dev, @fs, @rs)
self
end
alias add_row <<
def close
terminate
@ -709,7 +656,7 @@ class CSV
private
def initialize(dev)
raise RuntimeError.new('do not instantiate this class directly')
raise RuntimeError.new('Do not instanciate this class directly.')
end
def terminate
@ -719,12 +666,10 @@ class CSV
class BasicWriter < Writer
def initialize(str_or_writable, col_sep = ?,, row_sep = nil)
@col_sep = col_sep
@row_sep = row_sep
def initialize(str_or_writable, fs = ',', rs = nil)
@fs = fs
@rs = rs
@dev = str_or_writable
@dev.binmode if @dev.respond_to?(:binmode)
@close_on_terminate = false
end
@ -743,6 +688,7 @@ class CSV
end
end
private
# Buffered stream.
#
@ -756,7 +702,7 @@ class CSV
# end
#
# # define my own 'read' method.
# # CAUTION: Returning nil means EndOfStream.
# # CAUTION: Returning nil means EnfOfStream.
# def read(size)
# @s.read(size)
# end
@ -801,8 +747,7 @@ class CSV
# end
# end
#
class StreamBuf # pure virtual. (do not instantiate it directly)
class StreamBuf
# get a char or a partial string from the stream.
# idx: index of a string to specify a start point of a string to get.
# unlike String instance, idx < 0 returns nil.
@ -810,7 +755,7 @@ class CSV
# returns char at idx if n == nil.
# returns a partial string, from idx to (idx + n) if n != nil. at EOF,
# the string size could not equal to arg n.
def [](idx, n = nil)
def [](idx, n = nil)
if idx < 0
return nil
end
@ -838,11 +783,11 @@ class CSV
end
loc = my_offset + next_idx
if !n
return @buf_list[my_buf][loc] # Fixnum of char code.
return @buf_list[my_buf][loc] # Fixnum of char code.
elsif (loc + n - 1 < buf_size(my_buf))
return @buf_list[my_buf][loc, n] # String.
return @buf_list[my_buf][loc, n] # String.
else # should do loop insted of (tail) recursive call...
res = @buf_list[my_buf][loc, BufSize]
res = @buf_list[my_buf][loc, BufSize]
size_added = buf_size(my_buf) - loc
if size_added > 0
idx += size_added
@ -856,7 +801,7 @@ class CSV
end
end
alias get []
# drop a string from the stream.
# returns dropped size. at EOF, dropped size might not equals to arg n.
# Once you drop the head of the stream, access to the dropped part via []
@ -867,7 +812,7 @@ class CSV
end
size_dropped = 0
while (n > 0)
if (!@is_eos || (@cur_buf != @buf_tail_idx))
if !@is_eos or (@cur_buf != @buf_tail_idx)
if (@offset + n < buf_size(@cur_buf))
size_dropped += n
@offset += n
@ -888,11 +833,11 @@ class CSV
end
size_dropped
end
def is_eos?
return idx_is_eos?(0)
end
# WARN: Do not instantiate this class directly. Define your own class
# which derives this class and define 'read' instance method.
def initialize
@ -903,24 +848,23 @@ class CSV
add_buf
@cur_buf = @buf_tail_idx
end
protected
def terminate
while (rel_buf); end
end
# protected method 'read' must be defined in derived classes.
# CAUTION: Returning a string which size is not equal to 'size' means
# EndOfStream. When it is not at EOS, you must block the callee, try to
# EnfOfStream. When it is not at EOS, you must block the callee, try to
# read and return the sized string.
def read(size) # raise EOFError
raise NotImplementedError.new(
'method read must be defined in a derived class')
raise NotImplementedError.new('Method read must be defined in a derived class.')
end
private
def buf_size(idx)
@buf_list[idx].size
end
@ -948,7 +892,7 @@ class CSV
true
end
end
def rel_buf
if (@cur_buf < 0)
return false
@ -962,15 +906,14 @@ class CSV
return true
end
end
def idx_is_eos?(idx)
(@is_eos && ((@cur_buf < 0) || (@cur_buf == @buf_tail_idx)))
(@is_eos and ((@cur_buf < 0) or (@cur_buf == @buf_tail_idx)))
end
BufSize = 1024 * 8
end
# Buffered IO.
#
# EXAMPLE
@ -986,7 +929,7 @@ class CSV
@s = s
super()
end
def close
terminate
end
@ -996,7 +939,7 @@ class CSV
def read(size)
@s.read(size)
end
def terminate
super()
end

486
test/csv/test_csv.rb
Просмотреть файл

@ -1,5 +1,4 @@
require 'test/unit/testsuite'
require 'test/unit/testcase'
require 'test/unit'
require 'tempfile'
require 'fileutils'
@ -15,174 +14,20 @@ end
module CSVTestSupport
def d(data, is_null = false)
CSV::Cell.new(data.to_s, is_null)
end
end
class TestCSVCell < Test::Unit::TestCase
@@colData = ['', nil, true, false, 'foo', '!' * 1000]
def test_Cell_EQUAL # '=='
d1 = CSV::Cell.new('d', false)
d2 = CSV::Cell.new('d', false)
d3 = CSV::Cell.new('d', true)
d4 = CSV::Cell.new('d', true)
assert(d1 == d2, "Normal case.")
assert(d1 != d3, "RHS is null.")
assert(d4 != d1, "LHS is null.")
assert(d3 != d4, "Either is null.")
end
def test_Cell_match
d1 = CSV::Cell.new('d', false)
d2 = CSV::Cell.new('d', false)
d3 = CSV::Cell.new('d', true)
d4 = CSV::Cell.new('d', true)
assert(d1.match(d2), "Normal case.")
assert(!d1.match(d3), "RHS is null.")
assert(!d4.match(d1), "LHS is null.")
assert(d3.match(d4), "Either is null.")
end
def test_Cell_data
d = CSV::Cell.new()
@@colData.each do |v|
d.data = v
assert_equal(d.data, v, "Case: #{ v }.")
end
end
def test_Cell_data=
d = CSV::Cell.new()
@@colData.each do |v|
d.data = v
assert_equal(d.data, v, "Case: #{ v }.")
end
end
def test_Cell_is_null
d = CSV::Cell.new()
d.is_null = true
assert_equal(d.is_null, true, "Case: true.")
d.is_null = false
assert_equal(d.is_null, false, "Case: false.")
end
def test_Cell_is_null=
d = CSV::Cell.new()
d.is_null = true
assert_equal(d.is_null, true, "Case: true.")
d.is_null = false
assert_equal(d.is_null, false, "Case: false.")
end
def test_Cell_s_new
d1 = CSV::Cell.new()
assert_equal(d1.data, '', "Default: data.")
assert_equal(d1.is_null, true, "Default: is_null.")
@@colData.each do |v|
d = CSV::Cell.new(v)
assert_equal(d.data, v, "Data: #{ v }.")
end
d2 = CSV::Cell.new(nil, true)
assert_equal(d2.is_null, true, "Data: true.")
d3 = CSV::Cell.new(nil, false)
assert_equal(d3.is_null, false, "Data: false.")
end
def test_to_str
d = CSV::Cell.new("foo", false)
assert_equal("foo", d.to_str)
assert(/foo/ =~ d)
d = CSV::Cell.new("foo", true)
begin
d.to_str
assert(false)
rescue
# NoMethodError or NameError
assert(true)
end
end
def test_to_s
d = CSV::Cell.new("foo", false)
assert_equal("foo", d.to_s)
assert_equal("foo", "#{d}")
d = CSV::Cell.new("foo", true)
assert_equal("", d.to_s)
assert_equal("", "#{d}")
end
end
class TestCSVRow < Test::Unit::TestCase
include CSVTestSupport
def test_Row_s_match
c1 = CSV::Row[d(1), d(2), d(3)]
c2 = CSV::Row[d(1, false), d(2, false), d(3, false)]
assert(c1.match(c2), "Normal case.")
c1 = CSV::Row[d(1), d('foo', true), d(3)]
c2 = CSV::Row[d(1, false), d('bar', true), d(3, false)]
assert(c1.match(c2), "Either is null.")
c1 = CSV::Row[d(1), d('foo', true), d(3)]
c2 = CSV::Row[d(1, false), d('bar', false), d(3, false)]
assert(!c1.match(c2), "LHS is null.")
c1 = CSV::Row[d(1), d('foo'), d(3)]
c2 = CSV::Row[d(1, false), d('bar', true), d(3, false)]
assert(!c1.match(c2), "RHS is null.")
c1 = CSV::Row[d(1), d('', true), d(3)]
c2 = CSV::Row[d(1, false), d('', true), d(3, false)]
assert(c1.match(c2), "Either is null(empty data).")
c1 = CSV::Row[d(1), d('', true), d(3)]
c2 = CSV::Row[d(1, false), d('', false), d(3, false)]
assert(!c1.match(c2), "LHS is null(empty data).")
c1 = CSV::Row[d(1), d(''), d(3)]
c2 = CSV::Row[d(1, false), d('', true), d(3, false)]
assert(!c1.match(c2), "RHS is null(empty data).")
c1 = CSV::Row[]
c2 = CSV::Row[]
assert(c1.match(c2))
c1 = CSV::Row[]
c2 = CSV::Row[d(1)]
assert(!c1.match(c2))
end
def test_Row_to_a
r = CSV::Row[d(1), d(2), d(3)]
assert_equal(['1', '2', '3'], r.to_a, 'Normal case')
r = CSV::Row[d(1)]
assert_equal(['1'], r.to_a, '1 item')
r = CSV::Row[d(nil, true), d(2), d(3)]
assert_equal([nil, '2', '3'], r.to_a, 'Null in data')
r = CSV::Row[d(nil, true), d(nil, true), d(nil, true)]
assert_equal([nil, nil, nil], r.to_a, 'Nulls')
r = CSV::Row[d(nil, true)]
assert_equal([nil], r.to_a, '1 Null')
r = CSV::Row[]
assert_equal([], r.to_a, 'Empty')
def d(data)
data
end
end
class TestCSV < Test::Unit::TestCase
file = Tempfile.new("crlf")
file << "\n"
file.open
file.binmode
RSEP = file.read
file.close
include CSVTestSupport
class << self
@ -221,17 +66,17 @@ class TestCSV < Test::Unit::TestCase
}
@@fullCSVData = {
[d('', true)] => '',
[d(nil)] => '',
[d('')] => '""',
[d('', true), d('', true)] => ',',
[d('', true), d('', true), d('', true)] => ',,',
[d(nil), d(nil)] => ',',
[d(nil), d(nil), d(nil)] => ',,',
[d('foo')] => 'foo',
[d('foo'), d('bar')] => 'foo,bar',
[d('foo'), d('"bar"'), d('baz')] => 'foo,"""bar""",baz',
[d('foo'), d('foo,bar'), d('baz')] => 'foo,"foo,bar",baz',
[d('foo'), d('""'), d('baz')] => 'foo,"""""",baz',
[d('foo'), d(''), d('baz')] => 'foo,"",baz',
[d('foo'), d('', true), d('baz')] => 'foo,,baz',
[d('foo'), d(nil), d('baz')] => 'foo,,baz',
[d('foo'), d("\r"), d('baz')] => "foo,\"\r\",baz",
[d('foo'), d("\n"), d('baz')] => "foo,\"\n\",baz",
[d('foo'), d("\r\n"), d('baz')] => "foo,\"\r\n\",baz",
@ -259,7 +104,7 @@ class TestCSV < Test::Unit::TestCase
end
def sepConv(srcStr, srcSep, destSep, row_sep = nil)
rows = CSV::Row.new
rows = []
cols, idx = CSV.parse_row(srcStr, 0, rows, srcSep, row_sep)
destStr = ''
cols = CSV.generate_row(rows, rows.size, destStr, destSep, row_sep)
@ -278,13 +123,13 @@ public
@bomfile = File.join(@tmpdir, "bom.csv")
@macfile = File.join(@tmpdir, "mac.csv")
CSV.open(@infile, "w") do |writer|
CSV.open(@infile, "wb") do |writer|
@@fullCSVDataArray.each do |row|
writer.add_row(row)
end
end
CSV.open(@infiletsv, "w", ?\t) do |writer|
CSV.open(@infiletsv, "wb", ?\t) do |writer|
@@fullCSVDataArray.each do |row|
writer.add_row(row)
end
@ -317,11 +162,11 @@ public
first = true
ret = reader.each { |row|
if first
assert_instance_of(CSV::Row, row)
assert_instance_of(Array, row)
first = false
end
expected = expectedArray.shift
assert(row.match(expected))
assert_equal(expected, row)
}
assert_nil(ret, "Return is nil")
assert(expectedArray.empty?)
@ -352,10 +197,10 @@ public
@@fullCSVDataArray.each do |expected|
actual = reader.shift
if first
assert_instance_of(CSV::Row, actual)
assert_instance_of(Array, actual)
first = false
end
assert(actual.match(expected))
assert_equal(expected, actual)
checked += 1
end
assert(checked == @@fullCSVDataArray.size)
@ -445,7 +290,7 @@ public
file << "\"\r\n\",\"\r\",\"\n\"\r1,2,3"
file.close
file = File.open(@outfile, "r") # not "rb"
file = File.open(@outfile, "rb")
begin
reader = CSV::IOReader.new(file, ?,, ?\r)
assert_equal(["\r\n", "\r", "\n"], reader.shift.to_a)
@ -454,23 +299,34 @@ public
ensure
file.close
end
file = File.open(@outfile, "r") # not "rb"
begin
lfincell = (RSEP == "\n" ? "\r\n" : "\n")
reader = CSV::IOReader.new(file, ?,, ?\r)
assert_equal([lfincell, "\r", "\n"], reader.shift.to_a)
assert_equal(["1", "2", "3"], reader.shift.to_a)
reader.close
ensure
file.close
end
end
def test_Reader_s_parse
ret = CSV::Reader.parse("a,b,c") { |row|
assert_instance_of(CSV::Row, row, "Block parameter")
assert_instance_of(Array, row, "Block parameter")
}
assert_nil(ret, "Return is nil")
ret = CSV::Reader.parse("a;b;c", ?;) { |row|
assert_instance_of(CSV::Row, row, "Block parameter")
assert_instance_of(Array, row, "Block parameter")
}
file = Tempfile.new("in.csv")
file << "a,b,c"
file.open
ret = CSV::Reader.parse(file) { |row|
assert_instance_of(CSV::Row, row, "Block parameter")
assert_instance_of(Array, row, "Block parameter")
}
assert_nil(ret, "Return is nil")
@ -478,7 +334,7 @@ public
file << "a,b,c"
file.open
ret = CSV::Reader.parse(file, ?,) { |row|
assert_instance_of(CSV::Row, row, "Block parameter")
assert_instance_of(Array, row, "Block parameter")
}
# Illegal format.
@ -536,38 +392,38 @@ public
file.open
file.binmode
str = file.read
assert_equal("a,b,c\r\n,e,f\r\n,,\"\"\r\n", str, 'Normal')
assert_equal("a,b,c#{RSEP},e,f#{RSEP},,\"\"#{RSEP}", str, 'Normal')
file = Tempfile.new("out2.csv")
CSV::Writer.generate(file) do |writer|
ret = writer << [d('a'), d('b'), d('c')]
assert_instance_of(CSV::BasicWriter, ret, 'Return is self')
writer << [d(nil, true), d('e'), d('f')] << [d(nil, true), d(nil, true), d('')]
writer << [d(nil), d('e'), d('f')] << [d(nil), d(nil), d('')]
end
file.open
file.binmode
str = file.read
assert_equal("a,b,c\r\n,e,f\r\n,,\"\"\r\n", str, 'Normal')
assert_equal("a,b,c#{RSEP},e,f#{RSEP},,\"\"#{RSEP}", str, 'Normal')
end
def test_Writer_add_row
file = Tempfile.new("out.csv")
CSV::Writer.generate(file) do |writer|
ret = writer.add_row(
[d('a', false), d('b', false), d('c', false)])
[d('a'), d('b'), d('c')])
assert_instance_of(CSV::BasicWriter, ret, 'Return is self')
writer.add_row(
[d('dummy', true), d('e', false), d('f', false)]
[d(nil), d('e'), d('f')]
).add_row(
[d('a', true), d('b', true), d('', false)]
[d(nil), d(nil), d('')]
)
end
file.open
file.binmode
str = file.read
assert_equal("a,b,c\r\n,e,f\r\n,,\"\"\r\n", str, 'Normal')
assert_equal("a,b,c#{RSEP},e,f#{RSEP},,\"\"#{RSEP}", str, 'Normal')
end
def test_Writer_close
@ -606,7 +462,7 @@ public
file = File.open(@outfile, "rb")
str = file.read
file.close
assert_equal("\"\r\n\",\"\r\",\"\n\"\r1,2,3\r", str)
assert_equal("\"\r#{RSEP}\",\"\r\",\"#{RSEP}\"\r1,2,3\r", str)
end
#### CSV unit test
@ -633,12 +489,12 @@ public
reader.close
CSV.open(@infile, "r") do |row|
assert_instance_of(CSV::Row, row)
assert_instance_of(Array, row)
break
end
CSV.open(@infiletsv, "r", ?\t) do |row|
assert_instance_of(CSV::Row, row)
assert_instance_of(Array, row)
break
end
@ -686,12 +542,12 @@ public
reader.close
CSV.parse(@infile) do |row|
assert_instance_of(CSV::Row, row)
assert_instance_of(Array, row)
break
end
CSV.parse(@infiletsv, ?\t) do |row|
assert_instance_of(CSV::Row, row)
assert_instance_of(Array, row)
break
end
@ -776,12 +632,12 @@ public
@@simpleCSVData.each do |col, str|
buf = CSV.generate_line(col, ?;)
assert_equal(str + "\r\n", ssv2csv(buf))
assert_equal(str + "\n", ssv2csv(buf))
end
@@simpleCSVData.each do |col, str|
buf = CSV.generate_line(col, ?\t)
assert_equal(str + "\r\n", tsv2csv(buf))
assert_equal(str + "\n", tsv2csv(buf))
end
end
@ -789,17 +645,17 @@ public
buf = ''
cols = CSV.generate_row([], 0, buf)
assert_equal(0, cols)
assert_equal("\r\n", buf, "Extra boundary check.")
assert_equal("\n", buf, "Extra boundary check.")
buf = ''
cols = CSV.generate_row([], 0, buf, ?;)
assert_equal(0, cols)
assert_equal("\r\n", buf, "Extra boundary check.")
assert_equal("\n", buf, "Extra boundary check.")
buf = ''
cols = CSV.generate_row([], 0, buf, ?\t)
assert_equal(0, cols)
assert_equal("\r\n", buf, "Extra boundary check.")
assert_equal("\n", buf, "Extra boundary check.")
buf = ''
cols = CSV.generate_row([], 0, buf, ?\t, ?|)
@ -807,64 +663,64 @@ public
assert_equal("|", buf, "Extra boundary check.")
buf = ''
cols = CSV.generate_row([d(1)], 2, buf)
cols = CSV.generate_row([d('1')], 2, buf)
assert_equal('1,', buf)
buf = ''
cols = CSV.generate_row([d(1)], 2, buf, ?;)
cols = CSV.generate_row([d('1')], 2, buf, ?;)
assert_equal('1;', buf)
buf = ''
cols = CSV.generate_row([d(1)], 2, buf, ?\t)
cols = CSV.generate_row([d('1')], 2, buf, ?\t)
assert_equal("1\t", buf)
buf = ''
cols = CSV.generate_row([d(1)], 2, buf, ?\t, ?|)
cols = CSV.generate_row([d('1')], 2, buf, ?\t, ?|)
assert_equal("1\t", buf)
buf = ''
cols = CSV.generate_row([d(1), d(2)], 1, buf)
assert_equal("1\r\n", buf)
buf = ''
cols = CSV.generate_row([d(1), d(2)], 1, buf, ?;)
assert_equal("1\r\n", buf)
buf = ''
cols = CSV.generate_row([d(1), d(2)], 1, buf, ?\t)
assert_equal("1\r\n", buf)
buf = ''
cols = CSV.generate_row([d(1), d(2)], 1, buf, ?\t, ?\n)
cols = CSV.generate_row([d('1'), d('2')], 1, buf)
assert_equal("1\n", buf)
buf = ''
cols = CSV.generate_row([d(1), d(2)], 1, buf, ?\t, ?\r)
cols = CSV.generate_row([d('1'), d('2')], 1, buf, ?;)
assert_equal("1\n", buf)
buf = ''
cols = CSV.generate_row([d('1'), d('2')], 1, buf, ?\t)
assert_equal("1\n", buf)
buf = ''
cols = CSV.generate_row([d('1'), d('2')], 1, buf, ?\t, ?\n)
assert_equal("1\n", buf)
buf = ''
cols = CSV.generate_row([d('1'), d('2')], 1, buf, ?\t, ?\r)
assert_equal("1\r", buf)
buf = ''
cols = CSV.generate_row([d(1), d(2)], 1, buf, ?\t, ?|)
cols = CSV.generate_row([d('1'), d('2')], 1, buf, ?\t, ?|)
assert_equal("1|", buf)
@@fullCSVData.each do |col, str|
buf = ''
cols = CSV.generate_row(col, col.size, buf)
assert_equal(col.size, cols)
assert_equal(str + "\r\n", buf)
assert_equal(str + "\n", buf)
end
@@fullCSVData.each do |col, str|
buf = ''
cols = CSV.generate_row(col, col.size, buf, ?;)
assert_equal(col.size, cols)
assert_equal(str + "\r\n", ssv2csv(buf))
assert_equal(str + "\n", ssv2csv(buf))
end
@@fullCSVData.each do |col, str|
buf = ''
cols = CSV.generate_row(col, col.size, buf, ?\t)
assert_equal(col.size, cols)
assert_equal(str + "\r\n", tsv2csv(buf))
assert_equal(str + "\n", tsv2csv(buf))
end
# row separator
@ -889,7 +745,7 @@ public
colsToBe = 0
@@fullCSVData.each do |col, str|
cols += CSV.generate_row(col, col.size, buf)
toBe << str << "\r\n"
toBe << str << "\n"
colsToBe += col.size
end
assert_equal(colsToBe, cols)
@ -902,8 +758,8 @@ public
@@fullCSVData.each do |col, str|
lineBuf = ''
cols += CSV.generate_row(col, col.size, lineBuf, ?;)
buf << ssv2csv(lineBuf) << "\r\n"
toBe << ssv2csv(lineBuf) << "\r\n"
buf << ssv2csv(lineBuf) << "\n"
toBe << ssv2csv(lineBuf) << "\n"
colsToBe += col.size
end
assert_equal(colsToBe, cols)
@ -916,8 +772,8 @@ public
@@fullCSVData.each do |col, str|
lineBuf = ''
cols += CSV.generate_row(col, col.size, lineBuf, ?\t)
buf << tsv2csv(lineBuf) << "\r\n"
toBe << tsv2csv(lineBuf) << "\r\n"
buf << tsv2csv(lineBuf) << "\n"
toBe << tsv2csv(lineBuf) << "\n"
colsToBe += col.size
end
assert_equal(colsToBe, cols)
@ -941,7 +797,7 @@ public
def test_s_parse_line
@@simpleCSVData.each do |col, str|
row = CSV.parse_line(str)
assert_instance_of(CSV::Row, row)
assert_instance_of(Array, row)
assert_equal(col.size, row.size)
assert_equal(col, row)
end
@ -949,214 +805,214 @@ public
@@simpleCSVData.each do |col, str|
str = csv2ssv(str)
row = CSV.parse_line(str, ?;)
assert_instance_of(CSV::Row, row)
assert_equal(col.size, row.size)
assert_equal(col, row)
assert_instance_of(Array, row)
assert_equal(col.size, row.size, str.inspect)
assert_equal(col, row, str.inspect)
end
@@simpleCSVData.each do |col, str|
str = csv2tsv(str)
row = CSV.parse_line(str, ?\t)
assert_instance_of(CSV::Row, row)
assert_instance_of(Array, row)
assert_equal(col.size, row.size)
assert_equal(col, row)
end
# Illegal format.
buf = CSV::Row.new
buf = []
row = CSV.parse_line("a,b,\"c\"\ra")
assert_instance_of(CSV::Row, row)
assert_instance_of(Array, row)
assert_equal(0, row.size)
buf = CSV::Row.new
buf = Array.new
row = CSV.parse_line("a;b;\"c\"\ra", ?;)
assert_instance_of(CSV::Row, row)
assert_instance_of(Array, row)
assert_equal(0, row.size)
buf = CSV::Row.new
buf = Array.new
row = CSV.parse_line("a\tb\t\"c\"\ra", ?\t)
assert_instance_of(CSV::Row, row)
assert_instance_of(Array, row)
assert_equal(0, row.size)
row = CSV.parse_line("a,b\"")
assert_instance_of(CSV::Row, row)
assert_instance_of(Array, row)
assert_equal(0, row.size)
row = CSV.parse_line("a;b\"", ?;)
assert_instance_of(CSV::Row, row)
assert_instance_of(Array, row)
assert_equal(0, row.size)
row = CSV.parse_line("a\tb\"", ?\t)
assert_instance_of(CSV::Row, row)
assert_instance_of(Array, row)
assert_equal(0, row.size)
row = CSV.parse_line("\"a,b\"\r,")
assert_instance_of(CSV::Row, row)
assert_instance_of(Array, row)
assert_equal(0, row.size)
row = CSV.parse_line("\"a;b\"\r;", ?;)
assert_instance_of(CSV::Row, row)
assert_instance_of(Array, row)
assert_equal(0, row.size)
row = CSV.parse_line("\"a\tb\"\r\t", ?\t)
assert_instance_of(CSV::Row, row)
assert_instance_of(Array, row)
assert_equal(0, row.size)
row = CSV.parse_line("\"a,b\"\r\"")
assert_instance_of(CSV::Row, row)
assert_instance_of(Array, row)
assert_equal(0, row.size)
row = CSV.parse_line("\"a;b\"\r\"", ?;)
assert_instance_of(CSV::Row, row)
assert_instance_of(Array, row)
assert_equal(0, row.size)
row = CSV.parse_line("\"a\tb\"\r\"", ?\t)
assert_instance_of(CSV::Row, row)
assert_instance_of(Array, row)
assert_equal(0, row.size)
end
def test_s_parse_row
@@fullCSVData.each do |col, str|
buf = CSV::Row.new
buf = Array.new
cols, idx = CSV.parse_row(str + "\r\n", 0, buf)
assert_equal(cols, buf.size, "Reported size.")
assert_equal(col.size, buf.size, "Size.")
assert(buf.match(col))
assert_equal(col, buf, str.inspect)
buf = CSV::Row.new
cols, idx = CSV.parse_row(str + "\n", 0, buf)
buf = Array.new
cols, idx = CSV.parse_row(str + "\n", 0, buf, ?,, ?\n)
assert_equal(cols, buf.size, "Reported size.")
assert_equal(col.size, buf.size, "Size.")
assert(buf.match(col))
assert_equal(col, buf, str.inspect)
# separator: |
buf = CSV::Row.new
buf = Array.new
cols, idx = CSV.parse_row(str + "|", 0, buf, ?,)
assert(!buf.match(col))
buf = CSV::Row.new
assert_not_equal(col, buf)
buf = Array.new
cols, idx = CSV.parse_row(str + "|", 0, buf, ?,, ?|)
assert_equal(cols, buf.size, "Reported size.")
assert_equal(col.size, buf.size, "Size.")
assert(buf.match(col))
assert_equal(col, buf, str.inspect)
end
@@fullCSVData.each do |col, str|
str = csv2ssv(str)
buf = CSV::Row.new
buf = Array.new
cols, idx = CSV.parse_row(str + "\r\n", 0, buf, ?;)
assert_equal(cols, buf.size, "Reported size.")
assert_equal(col.size, buf.size, "Size.")
assert(buf.match(col))
assert_equal(col, buf, str)
end
@@fullCSVData.each do |col, str|
str = csv2tsv(str)
buf = CSV::Row.new
buf = Array.new
cols, idx = CSV.parse_row(str + "\r\n", 0, buf, ?\t)
assert_equal(cols, buf.size, "Reported size.")
assert_equal(col.size, buf.size, "Size.")
assert(buf.match(col))
assert_equal(col, buf, str)
end
@@fullCSVData.each do |col, str|
str = csv2tsv(str, ?|)
buf = CSV::Row.new
buf = Array.new
cols, idx = CSV.parse_row(str + "|", 0, buf, ?\t, ?|)
assert_equal(cols, buf.size, "Reported size.")
assert_equal(col.size, buf.size, "Size.")
assert(buf.match(col), str)
assert_equal(col, buf, str)
end
buf = CSV::Row.new
buf = Array.new
cols, idx = CSV.parse_row("a,b,\"c\r\"", 0, buf)
assert_equal(["a", "b", "c\r"], buf.to_a)
buf = CSV::Row.new
buf = Array.new
cols, idx = CSV.parse_row("a;b;\"c\r\"", 0, buf, ?;)
assert_equal(["a", "b", "c\r"], buf.to_a)
buf = CSV::Row.new
buf = Array.new
cols, idx = CSV.parse_row("a\tb\t\"c\r\"", 0, buf, ?\t)
assert_equal(["a", "b", "c\r"], buf.to_a)
buf = CSV::Row.new
cols, idx = CSV.parse_row("a,b,c\n", 0, buf)
buf = Array.new
cols, idx = CSV.parse_row("a,b,c\n", 0, buf, ?,, ?\n)
assert_equal(["a", "b", "c"], buf.to_a)
buf = CSV::Row.new
cols, idx = CSV.parse_row("a\tb\tc\n", 0, buf, ?\t)
buf = Array.new
cols, idx = CSV.parse_row("a\tb\tc\n", 0, buf, ?\t, ?\n)
assert_equal(["a", "b", "c"], buf.to_a)
# Illegal format.
buf = CSV::Row.new
buf = Array.new
cols, idx = CSV.parse_row("a,b,c\"", 0, buf)
assert_equal(0, cols, "Illegal format; unbalanced double-quote.")
buf = CSV::Row.new
buf = Array.new
cols, idx = CSV.parse_row("a;b;c\"", 0, buf, ?;)
assert_equal(0, cols, "Illegal format; unbalanced double-quote.")
buf = CSV::Row.new
buf = Array.new
cols, idx = CSV.parse_row("a,b,\"c\"\ra", 0, buf)
assert_equal(0, cols)
assert_equal(0, idx)
buf = CSV::Row.new
buf = Array.new
cols, idx = CSV.parse_row("a,b,\"c\"\ra", 0, buf, ?;)
assert_equal(0, cols)
assert_equal(0, idx)
buf = CSV::Row.new
buf = Array.new
cols, idx = CSV.parse_row("a,b\"", 0, buf)
assert_equal(0, cols)
assert_equal(0, idx)
buf = CSV::Row.new
buf = Array.new
cols, idx = CSV.parse_row("a;b\"", 0, buf, ?;)
assert_equal(0, cols)
assert_equal(0, idx)
buf = CSV::Row.new
buf = Array.new
cols, idx = CSV.parse_row("\"a,b\"\r,", 0, buf)
assert_equal(0, cols)
assert_equal(0, idx)
buf = CSV::Row.new
buf = Array.new
cols, idx = CSV.parse_row("a\r,", 0, buf)
assert_equal(0, cols)
assert_equal(0, idx)
buf = CSV::Row.new
buf = Array.new
cols, idx = CSV.parse_row("a\r", 0, buf)
assert_equal(0, cols)
assert_equal(0, idx)
buf = CSV::Row.new
buf = Array.new
cols, idx = CSV.parse_row("a\rbc", 0, buf)
assert_equal(0, cols)
assert_equal(0, idx)
buf = CSV::Row.new
buf = Array.new
cols, idx = CSV.parse_row("a\r\"\"", 0, buf)
assert_equal(0, cols)
assert_equal(0, idx)
buf = CSV::Row.new
buf = Array.new
cols, idx = CSV.parse_row("a\r\rabc,", 0, buf)
assert_equal(0, cols)
assert_equal(0, idx)
buf = CSV::Row.new
buf = Array.new
cols, idx = CSV.parse_row("\"a;b\"\r;", 0, buf, ?;)
assert_equal(0, cols)
assert_equal(0, idx)
buf = CSV::Row.new
buf = Array.new
cols, idx = CSV.parse_row("\"a,b\"\r\"", 0, buf)
assert_equal(0, cols)
assert_equal(0, idx)
buf = CSV::Row.new
buf = Array.new
cols, idx = CSV.parse_row("\"a;b\"\r\"", 0, buf, ?;)
assert_equal(0, cols)
assert_equal(0, idx)
@ -1168,11 +1024,11 @@ public
# String "" is not allowed.
next
end
buf = CSV::Row.new
buf = Array.new
cols, idx = CSV.parse_row(str, 0, buf)
assert_equal(col.size, cols, "Reported size.")
assert_equal(col.size, buf.size, "Size.")
assert(buf.match(col))
assert_equal(col, buf)
end
end
@ -1185,7 +1041,7 @@ public
end
idx = 0
cols = 0
parsed = CSV::Row.new
parsed = Array.new
parsedCols = 0
begin
cols, idx = CSV.parse_row(buf, idx, parsed)
@ -1193,7 +1049,7 @@ public
end while cols > 0
assert_equal(toBe.size, parsedCols)
assert_equal(toBe.size, parsed.size)
assert(parsed.match(toBe))
assert_equal(toBe, parsed)
buf = ''
toBe = []
@ -1203,15 +1059,15 @@ public
end
idx = 0
cols = 0
parsed = CSV::Row.new
parsed = Array.new
parsedCols = 0
begin
cols, idx = CSV.parse_row(buf, idx, parsed)
cols, idx = CSV.parse_row(buf, idx, parsed, ?,, ?\n)
parsedCols += cols
end while cols > 0
assert_equal(toBe.size, parsedCols)
assert_equal(toBe.size, parsed.size)
assert(parsed.match(toBe))
assert_equal(toBe, parsed)
buf = ''
toBe = []
@ -1223,15 +1079,15 @@ public
end
idx = 0
cols = 0
parsed = CSV::Row.new
parsed = Array.new
parsedCols = 0
begin
cols, idx = CSV.parse_row(buf, idx, parsed)
cols, idx = CSV.parse_row(buf, idx, parsed, ?,, ?\n)
parsedCols += cols
end while cols > 0
assert_equal(toBe.size, parsedCols)
assert_equal(toBe.size, parsed.size)
assert(parsed.match(toBe))
assert_equal(toBe, parsed)
buf = ''
toBe = []
@ -1241,7 +1097,7 @@ public
end
idx = 0
cols = 0
parsed = CSV::Row.new
parsed = []
parsedCols = 0
begin
cols, idx = CSV.parse_row(buf, idx, parsed, ?,, ?|)
@ -1249,7 +1105,7 @@ public
end while cols > 0
assert_equal(toBe.size, parsedCols)
assert_equal(toBe.size, parsed.size)
assert(parsed.match(toBe))
assert_equal(toBe, parsed)
end
def test_utf8
@ -1278,26 +1134,34 @@ public
rows = []
assert_raises(CSV::IllegalFormatError) do
CSV.open(@macfile, "r") do |row|
rows << row.to_a
rows << row.to_a
end
assert_equal([["Avenches", "aus Umgebung\r\"Bad Hersfeld", "Ausgrabung"]], rows)
end
rows = []
file = File.open(@macfile)
CSV::Reader.parse(file, ?,, ?\r) do |row|
rows << row.to_a
begin
CSV::Reader.parse(file, ?,, ?\r) do |row|
rows << row.to_a
end
assert_equal([["Avenches", "aus Umgebung"], ["Bad Hersfeld", "Ausgrabung"]], rows)
ensure
file.close
end
assert_equal([["Avenches", "aus Umgebung"], ["Bad Hersfeld", "Ausgrabung"]], rows)
file.close
rows = []
file = File.open(@macfile)
assert_raises(CSV::IllegalFormatError) do
CSV::Reader.parse(file, ?,) do |row|
rows << row.to_a
begin
assert_raises(CSV::IllegalFormatError) do
CSV::Reader.parse(file, ?,) do |row|
rows << row.to_a
end
assert_equal([["Avenches", "aus Umgebung\r\"Bad Hersfeld", "Ausgrabung"]], rows)
end
ensure
file.close
end
file.close
end
@ -1678,8 +1542,8 @@ public
#
def test_s_parseAndCreate
colSize = 8
csvStr = "foo,!!!foo!!!,!foo,bar!,!!!!!!,!!,,!\r!,!\r\n!\r\nNaHi,!!!Na!!!,!Na,Hi!,!\r.\n!,!\r\n\n!,!!!!,!\n!,!\r\n!".gsub!('!', '"')
csvStrTerminated = csvStr + "\r\n"
csvStr = "foo,!!!foo!!!,!foo,bar!,!!!!!!,!!,,!\r!,!\r\n!\nNaHi,!!!Na!!!,!Na,Hi!,!\r.\n!,!\r\n\n!,!!!!,!\n!,!\r\n!".gsub!('!', '"')
csvStrTerminated = csvStr + "\n"
myStr = csvStr.dup
res1 = []; res2 = []
@ -1708,19 +1572,9 @@ public
buf = ''
CSV::Writer.generate(buf) do |writer|
parsed.each do |row|
writer << row.collect { |e| e.is_null ? nil : e.data }
writer << row
end
end
assert_equal(csvStrTerminated, buf)
end
end
if $0 == __FILE__
suite = Test::Unit::TestSuite.new('CSV')
ObjectSpace.each_object(Class) do |klass|
suite << klass.suite if (Test::Unit::TestCase > klass)
end
require 'test/unit/ui/console/testrunner'
Test::Unit::UI::Console::TestRunner.run(suite).passed?
end