Tensorflow DecodeCSV field error when reading csv
0
I currently am trying to train a Tensorflow model and am having issues just reading in my very simple CSV file. I currently get the following error when running my training script:
InvalidArgumentError (see above for traceback): Expect 2 fields but have 1 in record 0
CSV file looks like this:
github,Awesome per directory history for ZSH
github,PHP class which implements the Elo rating system
github,Comic Sans Everything
I have checked both the training and validation set for "extra" commas that might break the delimitation during the read process, but I have found no such errors. Is there a way to figure out which line in my dataset is breaking the read function?
def read_dataset(prefix):
# use prefix to create filename
filename = 'gs://{}/{}*csv*'.format(BUCKET, prefix)
if prefix == 'train':
mode = tf.contrib.learn.ModeKeys.TRAIN
else:
print('EvalSet')
mode = tf.contrib.learn.ModeKeys.EVAL
# the actual input function passed to TensorFlow
def _input_fn():
# could be a path to one file or a file pattern.
input_file_names = tf.train.match_filenames_once(filename)
filename_queue = tf.train.string_input_producer(input_file_names, shuffle=True)
# read CSV
reader = tf.TextLineReader(skip_header_lines=0)
_, value = reader.read_up_to(filename_queue, num_records=BATCH_SIZE)
print(value)
#value = tf.train.shuffle_batch([value], BATCH_SIZE, capacity=10*BATCH_SIZE, min_after_dequeue=BATCH_SIZE, enqueue_many=True, allow_smaller_final_batch=False)
value_column = tf.expand_dims(value, -1)
columns = tf.decode_csv(value_column, record_defaults = DEFAULTS, field_delim=',', use_quote_delim=False, na_value="navalue")
features = dict(zip(CSV_COLUMNS, columns))
label = features.pop(LABEL_COLUMN)
# make targets numeric
table = tf.contrib.lookup.index_table_from_tensor(
mapping=tf.constant(TARGETS), num_oov_buckets=0, default_value=-1)
target = table.lookup(label)
return features, target
return _input_fn
python tensorflow
asked Nov 21 '18 at 13:06
chattrat423chattrat423
2631316
add a comment |
0
I currently am trying to train a Tensorflow model and am having issues just reading in my very simple CSV file. I currently get the following error when running my training script:
InvalidArgumentError (see above for traceback): Expect 2 fields but have 1 in record 0
CSV file looks like this:
github,Awesome per directory history for ZSH
github,PHP class which implements the Elo rating system
github,Comic Sans Everything
I have checked both the training and validation set for "extra" commas that might break the delimitation during the read process, but I have found no such errors. Is there a way to figure out which line in my dataset is breaking the read function?
def read_dataset(prefix):
# use prefix to create filename
filename = 'gs://{}/{}*csv*'.format(BUCKET, prefix)
if prefix == 'train':
mode = tf.contrib.learn.ModeKeys.TRAIN
else:
print('EvalSet')
mode = tf.contrib.learn.ModeKeys.EVAL
# the actual input function passed to TensorFlow
def _input_fn():
# could be a path to one file or a file pattern.
input_file_names = tf.train.match_filenames_once(filename)
filename_queue = tf.train.string_input_producer(input_file_names, shuffle=True)
# read CSV
reader = tf.TextLineReader(skip_header_lines=0)
_, value = reader.read_up_to(filename_queue, num_records=BATCH_SIZE)
print(value)
#value = tf.train.shuffle_batch([value], BATCH_SIZE, capacity=10*BATCH_SIZE, min_after_dequeue=BATCH_SIZE, enqueue_many=True, allow_smaller_final_batch=False)
value_column = tf.expand_dims(value, -1)
columns = tf.decode_csv(value_column, record_defaults = DEFAULTS, field_delim=',', use_quote_delim=False, na_value="navalue")
features = dict(zip(CSV_COLUMNS, columns))
label = features.pop(LABEL_COLUMN)
# make targets numeric
table = tf.contrib.lookup.index_table_from_tensor(
mapping=tf.constant(TARGETS), num_oov_buckets=0, default_value=-1)
target = table.lookup(label)
return features, target
return _input_fn
python tensorflow
asked Nov 21 '18 at 13:06
chattrat423chattrat423
2631316
add a comment |
0
0
0
I currently am trying to train a Tensorflow model and am having issues just reading in my very simple CSV file. I currently get the following error when running my training script:
InvalidArgumentError (see above for traceback): Expect 2 fields but have 1 in record 0
CSV file looks like this:
github,Awesome per directory history for ZSH
github,PHP class which implements the Elo rating system
github,Comic Sans Everything
I have checked both the training and validation set for "extra" commas that might break the delimitation during the read process, but I have found no such errors. Is there a way to figure out which line in my dataset is breaking the read function?
def read_dataset(prefix):
# use prefix to create filename
filename = 'gs://{}/{}*csv*'.format(BUCKET, prefix)
if prefix == 'train':
mode = tf.contrib.learn.ModeKeys.TRAIN
else:
print('EvalSet')
mode = tf.contrib.learn.ModeKeys.EVAL
# the actual input function passed to TensorFlow
def _input_fn():
# could be a path to one file or a file pattern.
input_file_names = tf.train.match_filenames_once(filename)
filename_queue = tf.train.string_input_producer(input_file_names, shuffle=True)
# read CSV
reader = tf.TextLineReader(skip_header_lines=0)
_, value = reader.read_up_to(filename_queue, num_records=BATCH_SIZE)
print(value)
#value = tf.train.shuffle_batch([value], BATCH_SIZE, capacity=10*BATCH_SIZE, min_after_dequeue=BATCH_SIZE, enqueue_many=True, allow_smaller_final_batch=False)
value_column = tf.expand_dims(value, -1)
columns = tf.decode_csv(value_column, record_defaults = DEFAULTS, field_delim=',', use_quote_delim=False, na_value="navalue")
features = dict(zip(CSV_COLUMNS, columns))
label = features.pop(LABEL_COLUMN)
# make targets numeric
table = tf.contrib.lookup.index_table_from_tensor(
mapping=tf.constant(TARGETS), num_oov_buckets=0, default_value=-1)
target = table.lookup(label)
return features, target
return _input_fn
python tensorflow
asked Nov 21 '18 at 13:06
chattrat423chattrat423
2631316
I currently am trying to train a Tensorflow model and am having issues just reading in my very simple CSV file. I currently get the following error when running my training script:
InvalidArgumentError (see above for traceback): Expect 2 fields but have 1 in record 0
CSV file looks like this:
github,Awesome per directory history for ZSH
github,PHP class which implements the Elo rating system
github,Comic Sans Everything
I have checked both the training and validation set for "extra" commas that might break the delimitation during the read process, but I have found no such errors. Is there a way to figure out which line in my dataset is breaking the read function?
def read_dataset(prefix):
# use prefix to create filename
filename = 'gs://{}/{}*csv*'.format(BUCKET, prefix)
if prefix == 'train':
mode = tf.contrib.learn.ModeKeys.TRAIN
else:
print('EvalSet')
mode = tf.contrib.learn.ModeKeys.EVAL
# the actual input function passed to TensorFlow
def _input_fn():
# could be a path to one file or a file pattern.
input_file_names = tf.train.match_filenames_once(filename)
filename_queue = tf.train.string_input_producer(input_file_names, shuffle=True)
# read CSV
reader = tf.TextLineReader(skip_header_lines=0)
_, value = reader.read_up_to(filename_queue, num_records=BATCH_SIZE)
print(value)
#value = tf.train.shuffle_batch([value], BATCH_SIZE, capacity=10*BATCH_SIZE, min_after_dequeue=BATCH_SIZE, enqueue_many=True, allow_smaller_final_batch=False)
value_column = tf.expand_dims(value, -1)
columns = tf.decode_csv(value_column, record_defaults = DEFAULTS, field_delim=',', use_quote_delim=False, na_value="navalue")
features = dict(zip(CSV_COLUMNS, columns))
label = features.pop(LABEL_COLUMN)
# make targets numeric
table = tf.contrib.lookup.index_table_from_tensor(
mapping=tf.constant(TARGETS), num_oov_buckets=0, default_value=-1)
target = table.lookup(label)
return features, target
return _input_fn
python tensorflow
python tensorflow
asked Nov 21 '18 at 13:06
chattrat423chattrat423
2631316
asked Nov 21 '18 at 13:06
chattrat423chattrat423
2631316
asked Nov 21 '18 at 13:06
chattrat423chattrat423
2631316
asked Nov 21 '18 at 13:06
chattrat423chattrat423
2631316
asked Nov 21 '18 at 13:06
chattrat423chattrat423
2631316
2631316
add a comment |
add a comment |
0
active
oldest
votes
Your Answer
StackExchange.ifUsing("editor", function () {
StackExchange.using("externalEditor", function () {
StackExchange.using("snippets", function () {
StackExchange.snippets.init();
});
});
}, "code-snippets");
StackExchange.ready(function() {
var channelOptions = {
tags: "".split(" "),
id: "1"
};
initTagRenderer("".split(" "), "".split(" "), channelOptions);
StackExchange.using("externalEditor", function() {
// Have to fire editor after snippets, if snippets enabled
if (StackExchange.settings.snippets.snippetsEnabled) {
StackExchange.using("snippets", function() {
createEditor();
});
}
else {
createEditor();
}
});
function createEditor() {
StackExchange.prepareEditor({
heartbeatType: 'answer',
autoActivateHeartbeat: false,
convertImagesToLinks: true,
noModals: true,
showLowRepImageUploadWarning: true,
reputationToPostImages: 10,
bindNavPrevention: true,
postfix: "",
imageUploader: {
brandingHtml: "Powered by u003ca class="icon-imgur-white" href="https://imgur.com/"u003eu003c/au003e",
contentPolicyHtml: "User contributions licensed under u003ca href="https://creativecommons.org/licenses/by-sa/3.0/"u003ecc by-sa 3.0 with attribution requiredu003c/au003e u003ca href="https://stackoverflow.com/legal/content-policy"u003e(content policy)u003c/au003e",
allowUrls: true
},
onDemand: true,
discardSelector: ".discard-answer"
,immediatelyShowMarkdownHelp:true
});
}
});
draft saved
draft discarded
Sign up or log in
StackExchange.ready(function () {
StackExchange.helpers.onClickDraftSave('#login-link');
});
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Post as a guest
Required, but never shown
StackExchange.ready(
function () {
StackExchange.openid.initPostLogin('.new-post-login', 'https%3a%2f%2fstackoverflow.com%2fquestions%2f53412722%2ftensorflow-decodecsv-field-error-when-reading-csv%23new-answer', 'question_page');
}
);
Post as a guest
Required, but never shown
0
active
oldest
votes
0
active
oldest
votes
active
oldest
votes
active
oldest
votes
draft saved
draft discarded
Thanks for contributing an answer to Stack Overflow!
- Please be sure to answer the question. Provide details and share your research!
But avoid …
- Asking for help, clarification, or responding to other answers.
- Making statements based on opinion; back them up with references or personal experience.
To learn more, see our tips on writing great answers.
draft saved
draft discarded
Sign up or log in
StackExchange.ready(function () {
StackExchange.helpers.onClickDraftSave('#login-link');
});
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Post as a guest
Required, but never shown
StackExchange.ready(
function () {
StackExchange.openid.initPostLogin('.new-post-login', 'https%3a%2f%2fstackoverflow.com%2fquestions%2f53412722%2ftensorflow-decodecsv-field-error-when-reading-csv%23new-answer', 'question_page');
}
);
Post as a guest
Required, but never shown
Sign up or log in
StackExchange.ready(function () {
StackExchange.helpers.onClickDraftSave('#login-link');
});
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Post as a guest
Required, but never shown
Sign up or log in
StackExchange.ready(function () {
StackExchange.helpers.onClickDraftSave('#login-link');
});
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Post as a guest
Required, but never shown
Sign up or log in
StackExchange.ready(function () {
StackExchange.helpers.onClickDraftSave('#login-link');
});
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Post as a guest
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
