Convert time object to datetime format in python pandas
I have a dataset of column name DateTime having dtype object.
df['DateTime'] = pd.to_datetime(df['DateTime'])
I have used the above code to convert to datetime format then did a split in the column to have Date and Time separately
df['date'] = df['DateTime'].dt.date
df['time'] = df['DateTime'].dt.time
but after the split the format changes to object type and while converting it to datetime it showing error for the time column name as: TypeError: is not convertible to datetime
How to convert it to datetime format the time column
python python-3.x pandas datetime dataframe
add a comment |
I have a dataset of column name DateTime having dtype object.
df['DateTime'] = pd.to_datetime(df['DateTime'])
I have used the above code to convert to datetime format then did a split in the column to have Date and Time separately
df['date'] = df['DateTime'].dt.date
df['time'] = df['DateTime'].dt.time
but after the split the format changes to object type and while converting it to datetime it showing error for the time column name as: TypeError: is not convertible to datetime
How to convert it to datetime format the time column
python python-3.x pandas datetime dataframe
What do you mean "proper time format"...? Please show the code you're using that produces TypeError: is not convertible to datetime
– Jon Clements♦
Nov 25 '18 at 18:02
The same code that I have showed to convert the DateTime i.edf['time'] = pd.to_datetime(df.['time'])
– Nadeem Haque
Nov 25 '18 at 18:11
@NadeemHaque - converting to string is necessary likedf['time'] = pd.to_datetime(df.['time'].astype(str))
but then is added some dates, because datetimes with no dates not exist.
– jezrael
Nov 25 '18 at 18:29
1
@jezrael I'm new to python so got confused.. thank you for the help
– Nadeem Haque
Nov 25 '18 at 18:59
add a comment |
I have a dataset of column name DateTime having dtype object.
df['DateTime'] = pd.to_datetime(df['DateTime'])
I have used the above code to convert to datetime format then did a split in the column to have Date and Time separately
df['date'] = df['DateTime'].dt.date
df['time'] = df['DateTime'].dt.time
but after the split the format changes to object type and while converting it to datetime it showing error for the time column name as: TypeError: is not convertible to datetime
How to convert it to datetime format the time column
python python-3.x pandas datetime dataframe
I have a dataset of column name DateTime having dtype object.
df['DateTime'] = pd.to_datetime(df['DateTime'])
I have used the above code to convert to datetime format then did a split in the column to have Date and Time separately
df['date'] = df['DateTime'].dt.date
df['time'] = df['DateTime'].dt.time
but after the split the format changes to object type and while converting it to datetime it showing error for the time column name as: TypeError: is not convertible to datetime
How to convert it to datetime format the time column
python python-3.x pandas datetime dataframe
python python-3.x pandas datetime dataframe
edited Nov 25 '18 at 18:30
Nadeem Haque
asked Nov 25 '18 at 17:59
Nadeem HaqueNadeem Haque
194
194
What do you mean "proper time format"...? Please show the code you're using that produces TypeError: is not convertible to datetime
– Jon Clements♦
Nov 25 '18 at 18:02
The same code that I have showed to convert the DateTime i.edf['time'] = pd.to_datetime(df.['time'])
– Nadeem Haque
Nov 25 '18 at 18:11
@NadeemHaque - converting to string is necessary likedf['time'] = pd.to_datetime(df.['time'].astype(str))
but then is added some dates, because datetimes with no dates not exist.
– jezrael
Nov 25 '18 at 18:29
1
@jezrael I'm new to python so got confused.. thank you for the help
– Nadeem Haque
Nov 25 '18 at 18:59
add a comment |
What do you mean "proper time format"...? Please show the code you're using that produces TypeError: is not convertible to datetime
– Jon Clements♦
Nov 25 '18 at 18:02
The same code that I have showed to convert the DateTime i.edf['time'] = pd.to_datetime(df.['time'])
– Nadeem Haque
Nov 25 '18 at 18:11
@NadeemHaque - converting to string is necessary likedf['time'] = pd.to_datetime(df.['time'].astype(str))
but then is added some dates, because datetimes with no dates not exist.
– jezrael
Nov 25 '18 at 18:29
1
@jezrael I'm new to python so got confused.. thank you for the help
– Nadeem Haque
Nov 25 '18 at 18:59
What do you mean "proper time format"...? Please show the code you're using that produces TypeError: is not convertible to datetime
– Jon Clements♦
Nov 25 '18 at 18:02
What do you mean "proper time format"...? Please show the code you're using that produces TypeError: is not convertible to datetime
– Jon Clements♦
Nov 25 '18 at 18:02
The same code that I have showed to convert the DateTime i.e
df['time'] = pd.to_datetime(df.['time'])
– Nadeem Haque
Nov 25 '18 at 18:11
The same code that I have showed to convert the DateTime i.e
df['time'] = pd.to_datetime(df.['time'])
– Nadeem Haque
Nov 25 '18 at 18:11
@NadeemHaque - converting to string is necessary like
df['time'] = pd.to_datetime(df.['time'].astype(str))
but then is added some dates, because datetimes with no dates not exist.– jezrael
Nov 25 '18 at 18:29
@NadeemHaque - converting to string is necessary like
df['time'] = pd.to_datetime(df.['time'].astype(str))
but then is added some dates, because datetimes with no dates not exist.– jezrael
Nov 25 '18 at 18:29
1
1
@jezrael I'm new to python so got confused.. thank you for the help
– Nadeem Haque
Nov 25 '18 at 18:59
@jezrael I'm new to python so got confused.. thank you for the help
– Nadeem Haque
Nov 25 '18 at 18:59
add a comment |
2 Answers
2
active
oldest
votes
You can use combine
in list comprehension with zip
:
df = pd.DataFrame({'DateTime': ['2011-01-01 12:48:20', '2014-01-01 12:30:45']})
df['DateTime'] = pd.to_datetime(df['DateTime'])
df['date'] = df['DateTime'].dt.date
df['time'] = df['DateTime'].dt.time
import datetime
df['new'] = [datetime.datetime.combine(a, b) for a, b in zip(df['date'], df['time'])]
print (df)
DateTime date time new
0 2011-01-01 12:48:20 2011-01-01 12:48:20 2011-01-01 12:48:20
1 2014-01-01 12:30:45 2014-01-01 12:30:45 2014-01-01 12:30:45
Or convert to strings, join together and convert again:
df['new'] = pd.to_datetime(df['date'].astype(str) + ' ' +df['time'].astype(str))
print (df)
DateTime date time new
0 2011-01-01 12:48:20 2011-01-01 12:48:20 2011-01-01 12:48:20
1 2014-01-01 12:30:45 2014-01-01 12:30:45 2014-01-01 12:30:45
But if use floor
for remove times with converting times to timedeltas then use +
only:
df['date'] = df['DateTime'].dt.floor('d')
df['time'] = pd.to_timedelta(df['DateTime'].dt.strftime('%H:%M:%S'))
df['new'] = df['date'] + df['time']
print (df)
DateTime date time new
0 2011-01-01 12:48:20 2011-01-01 12:48:20 2011-01-01 12:48:20
1 2014-01-01 12:30:45 2014-01-01 12:30:45 2014-01-01 12:30:45
add a comment |
How to convert it back to datetime format the time column
There appears to be a misunderstanding. Pandas datetime
series must include date and time components. This is non-negotiable. You can simply use pd.to_datetime
without specifying a date and use the default 1900-01-01
date:
# date from jezrael
print(pd.to_datetime(df['time'], format='%H:%M:%S'))
0 1900-01-01 12:48:20
1 1900-01-01 12:30:45
Name: time, dtype: datetime64[ns]
Or use another date component, for example today's date:
today = pd.Timestamp('today').strftime('%Y-%m-%d')
print(pd.to_datetime(today + ' ' + df['time'].astype(str)))
0 2018-11-25 12:48:20
1 2018-11-25 12:30:45
Name: time, dtype: datetime64[ns]
Or recombine from your date
and time
series:
print(pd.to_datetime(df['date'].astype(str) + ' ' + df['time'].astype(str)))
0 2011-01-01 12:48:20
1 2014-01-01 12:30:45
dtype: datetime64[ns]
thank you for the help
– Nadeem Haque
Nov 25 '18 at 19:01
@NadeemHaque, I'm glad my answer helped you. Please consider marking it as correct so it can help other people checking this question also.
– jpp
Nov 25 '18 at 20:18
add a comment |
Your Answer
StackExchange.ifUsing("editor", function () {
StackExchange.using("externalEditor", function () {
StackExchange.using("snippets", function () {
StackExchange.snippets.init();
});
});
}, "code-snippets");
StackExchange.ready(function() {
var channelOptions = {
tags: "".split(" "),
id: "1"
};
initTagRenderer("".split(" "), "".split(" "), channelOptions);
StackExchange.using("externalEditor", function() {
// Have to fire editor after snippets, if snippets enabled
if (StackExchange.settings.snippets.snippetsEnabled) {
StackExchange.using("snippets", function() {
createEditor();
});
}
else {
createEditor();
}
});
function createEditor() {
StackExchange.prepareEditor({
heartbeatType: 'answer',
autoActivateHeartbeat: false,
convertImagesToLinks: true,
noModals: true,
showLowRepImageUploadWarning: true,
reputationToPostImages: 10,
bindNavPrevention: true,
postfix: "",
imageUploader: {
brandingHtml: "Powered by u003ca class="icon-imgur-white" href="https://imgur.com/"u003eu003c/au003e",
contentPolicyHtml: "User contributions licensed under u003ca href="https://creativecommons.org/licenses/by-sa/3.0/"u003ecc by-sa 3.0 with attribution requiredu003c/au003e u003ca href="https://stackoverflow.com/legal/content-policy"u003e(content policy)u003c/au003e",
allowUrls: true
},
onDemand: true,
discardSelector: ".discard-answer"
,immediatelyShowMarkdownHelp:true
});
}
});
Sign up or log in
StackExchange.ready(function () {
StackExchange.helpers.onClickDraftSave('#login-link');
});
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Post as a guest
Required, but never shown
StackExchange.ready(
function () {
StackExchange.openid.initPostLogin('.new-post-login', 'https%3a%2f%2fstackoverflow.com%2fquestions%2f53470304%2fconvert-time-object-to-datetime-format-in-python-pandas%23new-answer', 'question_page');
}
);
Post as a guest
Required, but never shown
2 Answers
2
active
oldest
votes
2 Answers
2
active
oldest
votes
active
oldest
votes
active
oldest
votes
You can use combine
in list comprehension with zip
:
df = pd.DataFrame({'DateTime': ['2011-01-01 12:48:20', '2014-01-01 12:30:45']})
df['DateTime'] = pd.to_datetime(df['DateTime'])
df['date'] = df['DateTime'].dt.date
df['time'] = df['DateTime'].dt.time
import datetime
df['new'] = [datetime.datetime.combine(a, b) for a, b in zip(df['date'], df['time'])]
print (df)
DateTime date time new
0 2011-01-01 12:48:20 2011-01-01 12:48:20 2011-01-01 12:48:20
1 2014-01-01 12:30:45 2014-01-01 12:30:45 2014-01-01 12:30:45
Or convert to strings, join together and convert again:
df['new'] = pd.to_datetime(df['date'].astype(str) + ' ' +df['time'].astype(str))
print (df)
DateTime date time new
0 2011-01-01 12:48:20 2011-01-01 12:48:20 2011-01-01 12:48:20
1 2014-01-01 12:30:45 2014-01-01 12:30:45 2014-01-01 12:30:45
But if use floor
for remove times with converting times to timedeltas then use +
only:
df['date'] = df['DateTime'].dt.floor('d')
df['time'] = pd.to_timedelta(df['DateTime'].dt.strftime('%H:%M:%S'))
df['new'] = df['date'] + df['time']
print (df)
DateTime date time new
0 2011-01-01 12:48:20 2011-01-01 12:48:20 2011-01-01 12:48:20
1 2014-01-01 12:30:45 2014-01-01 12:30:45 2014-01-01 12:30:45
add a comment |
You can use combine
in list comprehension with zip
:
df = pd.DataFrame({'DateTime': ['2011-01-01 12:48:20', '2014-01-01 12:30:45']})
df['DateTime'] = pd.to_datetime(df['DateTime'])
df['date'] = df['DateTime'].dt.date
df['time'] = df['DateTime'].dt.time
import datetime
df['new'] = [datetime.datetime.combine(a, b) for a, b in zip(df['date'], df['time'])]
print (df)
DateTime date time new
0 2011-01-01 12:48:20 2011-01-01 12:48:20 2011-01-01 12:48:20
1 2014-01-01 12:30:45 2014-01-01 12:30:45 2014-01-01 12:30:45
Or convert to strings, join together and convert again:
df['new'] = pd.to_datetime(df['date'].astype(str) + ' ' +df['time'].astype(str))
print (df)
DateTime date time new
0 2011-01-01 12:48:20 2011-01-01 12:48:20 2011-01-01 12:48:20
1 2014-01-01 12:30:45 2014-01-01 12:30:45 2014-01-01 12:30:45
But if use floor
for remove times with converting times to timedeltas then use +
only:
df['date'] = df['DateTime'].dt.floor('d')
df['time'] = pd.to_timedelta(df['DateTime'].dt.strftime('%H:%M:%S'))
df['new'] = df['date'] + df['time']
print (df)
DateTime date time new
0 2011-01-01 12:48:20 2011-01-01 12:48:20 2011-01-01 12:48:20
1 2014-01-01 12:30:45 2014-01-01 12:30:45 2014-01-01 12:30:45
add a comment |
You can use combine
in list comprehension with zip
:
df = pd.DataFrame({'DateTime': ['2011-01-01 12:48:20', '2014-01-01 12:30:45']})
df['DateTime'] = pd.to_datetime(df['DateTime'])
df['date'] = df['DateTime'].dt.date
df['time'] = df['DateTime'].dt.time
import datetime
df['new'] = [datetime.datetime.combine(a, b) for a, b in zip(df['date'], df['time'])]
print (df)
DateTime date time new
0 2011-01-01 12:48:20 2011-01-01 12:48:20 2011-01-01 12:48:20
1 2014-01-01 12:30:45 2014-01-01 12:30:45 2014-01-01 12:30:45
Or convert to strings, join together and convert again:
df['new'] = pd.to_datetime(df['date'].astype(str) + ' ' +df['time'].astype(str))
print (df)
DateTime date time new
0 2011-01-01 12:48:20 2011-01-01 12:48:20 2011-01-01 12:48:20
1 2014-01-01 12:30:45 2014-01-01 12:30:45 2014-01-01 12:30:45
But if use floor
for remove times with converting times to timedeltas then use +
only:
df['date'] = df['DateTime'].dt.floor('d')
df['time'] = pd.to_timedelta(df['DateTime'].dt.strftime('%H:%M:%S'))
df['new'] = df['date'] + df['time']
print (df)
DateTime date time new
0 2011-01-01 12:48:20 2011-01-01 12:48:20 2011-01-01 12:48:20
1 2014-01-01 12:30:45 2014-01-01 12:30:45 2014-01-01 12:30:45
You can use combine
in list comprehension with zip
:
df = pd.DataFrame({'DateTime': ['2011-01-01 12:48:20', '2014-01-01 12:30:45']})
df['DateTime'] = pd.to_datetime(df['DateTime'])
df['date'] = df['DateTime'].dt.date
df['time'] = df['DateTime'].dt.time
import datetime
df['new'] = [datetime.datetime.combine(a, b) for a, b in zip(df['date'], df['time'])]
print (df)
DateTime date time new
0 2011-01-01 12:48:20 2011-01-01 12:48:20 2011-01-01 12:48:20
1 2014-01-01 12:30:45 2014-01-01 12:30:45 2014-01-01 12:30:45
Or convert to strings, join together and convert again:
df['new'] = pd.to_datetime(df['date'].astype(str) + ' ' +df['time'].astype(str))
print (df)
DateTime date time new
0 2011-01-01 12:48:20 2011-01-01 12:48:20 2011-01-01 12:48:20
1 2014-01-01 12:30:45 2014-01-01 12:30:45 2014-01-01 12:30:45
But if use floor
for remove times with converting times to timedeltas then use +
only:
df['date'] = df['DateTime'].dt.floor('d')
df['time'] = pd.to_timedelta(df['DateTime'].dt.strftime('%H:%M:%S'))
df['new'] = df['date'] + df['time']
print (df)
DateTime date time new
0 2011-01-01 12:48:20 2011-01-01 12:48:20 2011-01-01 12:48:20
1 2014-01-01 12:30:45 2014-01-01 12:30:45 2014-01-01 12:30:45
answered Nov 25 '18 at 18:10
jezraeljezrael
347k25304379
347k25304379
add a comment |
add a comment |
How to convert it back to datetime format the time column
There appears to be a misunderstanding. Pandas datetime
series must include date and time components. This is non-negotiable. You can simply use pd.to_datetime
without specifying a date and use the default 1900-01-01
date:
# date from jezrael
print(pd.to_datetime(df['time'], format='%H:%M:%S'))
0 1900-01-01 12:48:20
1 1900-01-01 12:30:45
Name: time, dtype: datetime64[ns]
Or use another date component, for example today's date:
today = pd.Timestamp('today').strftime('%Y-%m-%d')
print(pd.to_datetime(today + ' ' + df['time'].astype(str)))
0 2018-11-25 12:48:20
1 2018-11-25 12:30:45
Name: time, dtype: datetime64[ns]
Or recombine from your date
and time
series:
print(pd.to_datetime(df['date'].astype(str) + ' ' + df['time'].astype(str)))
0 2011-01-01 12:48:20
1 2014-01-01 12:30:45
dtype: datetime64[ns]
thank you for the help
– Nadeem Haque
Nov 25 '18 at 19:01
@NadeemHaque, I'm glad my answer helped you. Please consider marking it as correct so it can help other people checking this question also.
– jpp
Nov 25 '18 at 20:18
add a comment |
How to convert it back to datetime format the time column
There appears to be a misunderstanding. Pandas datetime
series must include date and time components. This is non-negotiable. You can simply use pd.to_datetime
without specifying a date and use the default 1900-01-01
date:
# date from jezrael
print(pd.to_datetime(df['time'], format='%H:%M:%S'))
0 1900-01-01 12:48:20
1 1900-01-01 12:30:45
Name: time, dtype: datetime64[ns]
Or use another date component, for example today's date:
today = pd.Timestamp('today').strftime('%Y-%m-%d')
print(pd.to_datetime(today + ' ' + df['time'].astype(str)))
0 2018-11-25 12:48:20
1 2018-11-25 12:30:45
Name: time, dtype: datetime64[ns]
Or recombine from your date
and time
series:
print(pd.to_datetime(df['date'].astype(str) + ' ' + df['time'].astype(str)))
0 2011-01-01 12:48:20
1 2014-01-01 12:30:45
dtype: datetime64[ns]
thank you for the help
– Nadeem Haque
Nov 25 '18 at 19:01
@NadeemHaque, I'm glad my answer helped you. Please consider marking it as correct so it can help other people checking this question also.
– jpp
Nov 25 '18 at 20:18
add a comment |
How to convert it back to datetime format the time column
There appears to be a misunderstanding. Pandas datetime
series must include date and time components. This is non-negotiable. You can simply use pd.to_datetime
without specifying a date and use the default 1900-01-01
date:
# date from jezrael
print(pd.to_datetime(df['time'], format='%H:%M:%S'))
0 1900-01-01 12:48:20
1 1900-01-01 12:30:45
Name: time, dtype: datetime64[ns]
Or use another date component, for example today's date:
today = pd.Timestamp('today').strftime('%Y-%m-%d')
print(pd.to_datetime(today + ' ' + df['time'].astype(str)))
0 2018-11-25 12:48:20
1 2018-11-25 12:30:45
Name: time, dtype: datetime64[ns]
Or recombine from your date
and time
series:
print(pd.to_datetime(df['date'].astype(str) + ' ' + df['time'].astype(str)))
0 2011-01-01 12:48:20
1 2014-01-01 12:30:45
dtype: datetime64[ns]
How to convert it back to datetime format the time column
There appears to be a misunderstanding. Pandas datetime
series must include date and time components. This is non-negotiable. You can simply use pd.to_datetime
without specifying a date and use the default 1900-01-01
date:
# date from jezrael
print(pd.to_datetime(df['time'], format='%H:%M:%S'))
0 1900-01-01 12:48:20
1 1900-01-01 12:30:45
Name: time, dtype: datetime64[ns]
Or use another date component, for example today's date:
today = pd.Timestamp('today').strftime('%Y-%m-%d')
print(pd.to_datetime(today + ' ' + df['time'].astype(str)))
0 2018-11-25 12:48:20
1 2018-11-25 12:30:45
Name: time, dtype: datetime64[ns]
Or recombine from your date
and time
series:
print(pd.to_datetime(df['date'].astype(str) + ' ' + df['time'].astype(str)))
0 2011-01-01 12:48:20
1 2014-01-01 12:30:45
dtype: datetime64[ns]
answered Nov 25 '18 at 18:28
jppjpp
102k2165115
102k2165115
thank you for the help
– Nadeem Haque
Nov 25 '18 at 19:01
@NadeemHaque, I'm glad my answer helped you. Please consider marking it as correct so it can help other people checking this question also.
– jpp
Nov 25 '18 at 20:18
add a comment |
thank you for the help
– Nadeem Haque
Nov 25 '18 at 19:01
@NadeemHaque, I'm glad my answer helped you. Please consider marking it as correct so it can help other people checking this question also.
– jpp
Nov 25 '18 at 20:18
thank you for the help
– Nadeem Haque
Nov 25 '18 at 19:01
thank you for the help
– Nadeem Haque
Nov 25 '18 at 19:01
@NadeemHaque, I'm glad my answer helped you. Please consider marking it as correct so it can help other people checking this question also.
– jpp
Nov 25 '18 at 20:18
@NadeemHaque, I'm glad my answer helped you. Please consider marking it as correct so it can help other people checking this question also.
– jpp
Nov 25 '18 at 20:18
add a comment |
Thanks for contributing an answer to Stack Overflow!
- Please be sure to answer the question. Provide details and share your research!
But avoid …
- Asking for help, clarification, or responding to other answers.
- Making statements based on opinion; back them up with references or personal experience.
To learn more, see our tips on writing great answers.
Sign up or log in
StackExchange.ready(function () {
StackExchange.helpers.onClickDraftSave('#login-link');
});
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Post as a guest
Required, but never shown
StackExchange.ready(
function () {
StackExchange.openid.initPostLogin('.new-post-login', 'https%3a%2f%2fstackoverflow.com%2fquestions%2f53470304%2fconvert-time-object-to-datetime-format-in-python-pandas%23new-answer', 'question_page');
}
);
Post as a guest
Required, but never shown
Sign up or log in
StackExchange.ready(function () {
StackExchange.helpers.onClickDraftSave('#login-link');
});
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Post as a guest
Required, but never shown
Sign up or log in
StackExchange.ready(function () {
StackExchange.helpers.onClickDraftSave('#login-link');
});
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Post as a guest
Required, but never shown
Sign up or log in
StackExchange.ready(function () {
StackExchange.helpers.onClickDraftSave('#login-link');
});
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Post as a guest
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
What do you mean "proper time format"...? Please show the code you're using that produces TypeError: is not convertible to datetime
– Jon Clements♦
Nov 25 '18 at 18:02
The same code that I have showed to convert the DateTime i.e
df['time'] = pd.to_datetime(df.['time'])
– Nadeem Haque
Nov 25 '18 at 18:11
@NadeemHaque - converting to string is necessary like
df['time'] = pd.to_datetime(df.['time'].astype(str))
but then is added some dates, because datetimes with no dates not exist.– jezrael
Nov 25 '18 at 18:29
1
@jezrael I'm new to python so got confused.. thank you for the help
– Nadeem Haque
Nov 25 '18 at 18:59