Ggtcf

Return the Closest Prime Number

Is "for causing autism in X" grammatical?

Is it possible to search for a directory/file combination?

How did people program for Consoles with multiple CPUs?

What flight has the highest ratio of time difference to flight time?

Indicator light circuit

Why does the UK parliament need a vote on the political declaration?

Unreliable Magic - Is it worth it?

If/When UK leaves the EU, can a future goverment conduct a referendum to join the EU?

Help understanding this unsettling image of Titan, Epimetheus, and Saturn's rings?

Are there any unintended negative consequences to allowing PCs to gain multiple levels at once in a short milestone-XP game?

Skipping indices in a product

How do I go from 300 unfinished/half written blog posts, to published posts?

What is the purpose of the Evocation wizard's Potent Cantrip feature?

Is micro rebar a better way to reinforce concrete than rebar?

If a black hole is created from light, can this black hole then move at speed of light?

Won the lottery - how do I keep the money?

I believe this to be a fraud - hired, then asked to cash check and send cash as Bitcoin

Why is the US ranked as #45 in Press Freedom ratings, despite its extremely permissive free speech laws?

Novel about a guy who is possessed by the divine essence and the world ends?

What is "(CFMCC)" on an ILS approach chart?

Why don't programming languages automatically manage the synchronous/asynchronous problem?

Interfacing a button to MCU (and PC) with 50m long cable

Between two walls

pandas.DataFrame.replace() got “OverflowError: Python int too large to convert to C long” [duplicate]

1

This question already has an answer here:

• 
  OverflowError: Python int too large to convert to C long
  
  3 answers
  
  

When I use pandas.DataFrame.replace(dict) to convert user_id string to integer, I receive:

"OverflowError: Python int too large to convert to C long".

sample code:

import pandas as pd
x = 'user_id':['100000715097692381911', 
 '100003840837471130074'], 
 'item_id': [1, 2]
 
dfx = pd.DataFrame(x)
dfx['user_id'].replace(
 
 '100000715097692381911': 0, 
 '100003840837471130074': 1
 , inplace=True)

I don't understand why this is duplicated. I think this is a problem of pandas taking str type as integers. I didn't load those big id numbers as integer but as string. Well, if I prepend an character to 'user_id' string, like 's100000715097692381911', it will not report OverflowError.

python python-3.x pandas dataframe

share|improve this question

edited Mar 8 at 15:01

Weihao Wang

asked Mar 8 at 14:23

Weihao WangWeihao Wang

112

marked as duplicate by jezrael dataframe
Users with the  dataframe badge can single-handedly close dataframe questions as duplicates and reopen them as needed.

StackExchange.ready(function()
if (StackExchange.options.isMobile) return;

$('.dupe-hammer-message-hover:not(.hover-bound)').each(function()
var $hover = $(this).addClass('hover-bound'),
$msg = $hover.siblings('.dupe-hammer-message');

$hover.hover(
function()
$hover.showInfoMessage('',
messageElement: $msg.clone().show(),
transient: false,
position: my: 'bottom left', at: 'top center', offsetTop: -7 ,
dismissable: false,
relativeToBody: true
);
,
function()
StackExchange.helpers.removeMessages();

);
);
);
Mar 8 at 14:36

This question has been asked before and already has an answer. If those answers do not fully address your question, please ask a new question.

• 
  
  
  

  
  

  
  
  
  
  
  

  
  
  @Aeossa I searched before asking, and it seems the link you post can't solve my problem. I think maybe pandas somehow take the user_id string as a large integer? But dfx.dtypes shows its type is object
  
  – Weihao Wang
  Mar 8 at 14:32
  
  

  
  
  
  

add a comment | 

1

This question already has an answer here:

• 
  OverflowError: Python int too large to convert to C long
  
  3 answers
  
  

When I use pandas.DataFrame.replace(dict) to convert user_id string to integer, I receive:

"OverflowError: Python int too large to convert to C long".

sample code:

import pandas as pd
x = 'user_id':['100000715097692381911', 
 '100003840837471130074'], 
 'item_id': [1, 2]
 
dfx = pd.DataFrame(x)
dfx['user_id'].replace(
 
 '100000715097692381911': 0, 
 '100003840837471130074': 1
 , inplace=True)

I don't understand why this is duplicated. I think this is a problem of pandas taking str type as integers. I didn't load those big id numbers as integer but as string. Well, if I prepend an character to 'user_id' string, like 's100000715097692381911', it will not report OverflowError.

python python-3.x pandas dataframe

share|improve this question

edited Mar 8 at 15:01

Weihao Wang

asked Mar 8 at 14:23

Weihao WangWeihao Wang

112

marked as duplicate by jezrael dataframe
Users with the  dataframe badge can single-handedly close dataframe questions as duplicates and reopen them as needed.

StackExchange.ready(function()
if (StackExchange.options.isMobile) return;

$('.dupe-hammer-message-hover:not(.hover-bound)').each(function()
var $hover = $(this).addClass('hover-bound'),
$msg = $hover.siblings('.dupe-hammer-message');

$hover.hover(
function()
$hover.showInfoMessage('',
messageElement: $msg.clone().show(),
transient: false,
position: my: 'bottom left', at: 'top center', offsetTop: -7 ,
dismissable: false,
relativeToBody: true
);
,
function()
StackExchange.helpers.removeMessages();

);
);
);
Mar 8 at 14:36

This question has been asked before and already has an answer. If those answers do not fully address your question, please ask a new question.

• 
  
  
  

  
  

  
  
  
  
  
  

  
  
  @Aeossa I searched before asking, and it seems the link you post can't solve my problem. I think maybe pandas somehow take the user_id string as a large integer? But dfx.dtypes shows its type is object
  
  – Weihao Wang
  Mar 8 at 14:32
  
  

  
  
  
  

add a comment | 

This question already has an answer here:

• 
  OverflowError: Python int too large to convert to C long
  
  3 answers
  
  

When I use pandas.DataFrame.replace(dict) to convert user_id string to integer, I receive:

"OverflowError: Python int too large to convert to C long".

sample code:

import pandas as pd
x = 'user_id':['100000715097692381911', 
 '100003840837471130074'], 
 'item_id': [1, 2]
 
dfx = pd.DataFrame(x)
dfx['user_id'].replace(
 
 '100000715097692381911': 0, 
 '100003840837471130074': 1
 , inplace=True)

I don't understand why this is duplicated. I think this is a problem of pandas taking str type as integers. I didn't load those big id numbers as integer but as string. Well, if I prepend an character to 'user_id' string, like 's100000715097692381911', it will not report OverflowError.

python python-3.x pandas dataframe

share|improve this question

edited Mar 8 at 15:01

Weihao Wang

asked Mar 8 at 14:23

Weihao WangWeihao Wang

112

import pandas as pd
x = 'user_id':['100000715097692381911', 
 '100003840837471130074'], 
 'item_id': [1, 2]
 
dfx = pd.DataFrame(x)
dfx['user_id'].replace(
 
 '100000715097692381911': 0, 
 '100003840837471130074': 1
 , inplace=True)

share|improve this question

edited Mar 8 at 15:01

Weihao Wang

asked Mar 8 at 14:23

Weihao WangWeihao Wang

112

share|improve this question

edited Mar 8 at 15:01

Weihao Wang

asked Mar 8 at 14:23

Weihao WangWeihao Wang

112

asked Mar 8 at 14:23

Weihao WangWeihao Wang

112

asked Mar 8 at 14:23

Weihao WangWeihao Wang

112

1 Answer
1

active

oldest

votes

1

In C, a long is 4 bytes and can only store values between -2,147,483,648 and 2,147,483,647.

To answer your other question, a string in C is stored as a char array, and so it's memory space is 1 byte for each char, plus the size of the terminating pointer. This means a python string in C won't cause an overflow, but a large integer will.

Source: https://www.tutorialspoint.com/cprogramming/c_data_types.htm

share|improve this answer

answered Mar 8 at 14:35

Luke NingLuke Ning

926

• 
  
  
  

  
  

  
  
  
  
  
  

  
  
  Thanks, but I didn't load those big numbers as integers. You can see that they are string type in my code.
  
  – Weihao Wang
  Mar 8 at 14:44
  

  
  
  
  


• 
  
  
  

  
  

  
  
  
  
  
  

  
  
  @WeihaoWang Loading the big numbers as ints might help you. Try letting the big numbers be ints, and change your df definition to dfx = pd.DataFrame(x, dtype=str) and then changing your replace to look for int keys instead
  
  – Luke Ning
  Mar 8 at 14:51
  
  

  
  
  
  


• 
  
  
  

  
  

  
  
  
  
  
  

  
  
  I tried your suggestion, it doesn't work. Thank you anyway. Could it be a bug of pandas?
  
  – Weihao Wang
  Mar 8 at 14:58
  

  
  
  
  


• 
  
  
  

  
  

  
  
  
  
  
  

  
  
  @WeihaoWang really? it worked for me. Are you still getting an overflow error?
  
  – Luke Ning
  Mar 8 at 15:02
  

  
  
  
  


• 
  
  
  

  
  

  
  
  
  
  
  

  
  
  uh.. it didn't report overflow, but the elements are not replaced with correct values.
  
  – Weihao Wang
  Mar 8 at 15:09
  

  
  
  
  

 | 
show 4 more comments

1

In C, a long is 4 bytes and can only store values between -2,147,483,648 and 2,147,483,647.

To answer your other question, a string in C is stored as a char array, and so it's memory space is 1 byte for each char, plus the size of the terminating pointer. This means a python string in C won't cause an overflow, but a large integer will.

Source: https://www.tutorialspoint.com/cprogramming/c_data_types.htm

share|improve this answer

answered Mar 8 at 14:35

Luke NingLuke Ning

926

• 
  
  
  

  
  

  
  
  
  
  
  

  
  
  Thanks, but I didn't load those big numbers as integers. You can see that they are string type in my code.
  
  – Weihao Wang
  Mar 8 at 14:44
  

  
  
  
  


• 
  
  
  

  
  

  
  
  
  
  
  

  
  
  @WeihaoWang Loading the big numbers as ints might help you. Try letting the big numbers be ints, and change your df definition to dfx = pd.DataFrame(x, dtype=str) and then changing your replace to look for int keys instead
  
  – Luke Ning
  Mar 8 at 14:51
  
  

  
  
  
  


• 
  
  
  

  
  

  
  
  
  
  
  

  
  
  I tried your suggestion, it doesn't work. Thank you anyway. Could it be a bug of pandas?
  
  – Weihao Wang
  Mar 8 at 14:58
  

  
  
  
  


• 
  
  
  

  
  

  
  
  
  
  
  

  
  
  @WeihaoWang really? it worked for me. Are you still getting an overflow error?
  
  – Luke Ning
  Mar 8 at 15:02
  

  
  
  
  


• 
  
  
  

  
  

  
  
  
  
  
  

  
  
  uh.. it didn't report overflow, but the elements are not replaced with correct values.
  
  – Weihao Wang
  Mar 8 at 15:09
  

  
  
  
  

 | 
show 4 more comments

1

In C, a long is 4 bytes and can only store values between -2,147,483,648 and 2,147,483,647.

To answer your other question, a string in C is stored as a char array, and so it's memory space is 1 byte for each char, plus the size of the terminating pointer. This means a python string in C won't cause an overflow, but a large integer will.

Source: https://www.tutorialspoint.com/cprogramming/c_data_types.htm

share|improve this answer

answered Mar 8 at 14:35

Luke NingLuke Ning

926

• 
  
  
  

  
  

  
  
  
  
  
  

  
  
  Thanks, but I didn't load those big numbers as integers. You can see that they are string type in my code.
  
  – Weihao Wang
  Mar 8 at 14:44
  

  
  
  
  


• 
  
  
  

  
  

  
  
  
  
  
  

  
  
  @WeihaoWang Loading the big numbers as ints might help you. Try letting the big numbers be ints, and change your df definition to dfx = pd.DataFrame(x, dtype=str) and then changing your replace to look for int keys instead
  
  – Luke Ning
  Mar 8 at 14:51
  
  

  
  
  
  


• 
  
  
  

  
  

  
  
  
  
  
  

  
  
  I tried your suggestion, it doesn't work. Thank you anyway. Could it be a bug of pandas?
  
  – Weihao Wang
  Mar 8 at 14:58
  

  
  
  
  


• 
  
  
  

  
  

  
  
  
  
  
  

  
  
  @WeihaoWang really? it worked for me. Are you still getting an overflow error?
  
  – Luke Ning
  Mar 8 at 15:02
  

  
  
  
  


• 
  
  
  

  
  

  
  
  
  
  
  

  
  
  uh.. it didn't report overflow, but the elements are not replaced with correct values.
  
  – Weihao Wang
  Mar 8 at 15:09
  

  
  
  
  

 | 
show 4 more comments

In C, a long is 4 bytes and can only store values between -2,147,483,648 and 2,147,483,647.

To answer your other question, a string in C is stored as a char array, and so it's memory space is 1 byte for each char, plus the size of the terminating pointer. This means a python string in C won't cause an overflow, but a large integer will.

Source: https://www.tutorialspoint.com/cprogramming/c_data_types.htm

share|improve this answer

answered Mar 8 at 14:35

Luke NingLuke Ning

926

share|improve this answer

answered Mar 8 at 14:35

Luke NingLuke Ning

926

answered Mar 8 at 14:35

Luke NingLuke Ning

926

answered Mar 8 at 14:35

Luke NingLuke Ning

926