Have a huge data set and I need to analyze it (financial tools)Best way to construct a pandas.DataFrame composed of different chunksHow to remove day from datetime index in pandas?“Large data” work flows using pandasFilter data to get only first day of the month rowstime interval partitioned by 2 fields in pandasPython Pandas - Get Attributes Associated With Consecutive datetimeDaily High/Low on intraday dataPandas DataFrame and DateTimeIndexFor each day, subtract last time row from the first time row with PythonCreate plots of DataFrames using Python
Exposing a company lying about themselves in a tightly knit industry (videogames) : Is my career at risk on the long run?
A seasonal riddle
Does capillary rise violate hydrostatic paradox?
Taking the numerator and the denominator
How can I, as DM, avoid the Conga Line of Death occurring when implementing some form of flanking rule?
Strange behavior in TikZ draw command
Derivative of an interpolated function
Asserting that Atheism and Theism are both faith based positions
Why is participating in the European Parliamentary elections used as a threat?
Why didn’t Eve recognize the little cockroach as a living organism?
Sort with assumptions
Is there a distance limit for minecart tracks?
Hashing password to increase entropy
Extract substring according to regexp with sed or grep
Capacitor electron flow
Why is "la Gestapo" feminine?
PTIJ: Which Dr. Seuss books should one obtain?
If the Dominion rule using their Jem'Hadar troops, why is their life expectancy so low?
What is this high flying aircraft over Pennsylvania?
Highest stage count that are used one right after the other?
categorizing a variable turns it from insignificant to significant
Calculate Pi using Monte Carlo
Offset in split text content
Error in master's thesis, I do not know what to do
Have a huge data set and I need to analyze it (financial tools)
Best way to construct a pandas.DataFrame composed of different chunksHow to remove day from datetime index in pandas?“Large data” work flows using pandasFilter data to get only first day of the month rowstime interval partitioned by 2 fields in pandasPython Pandas - Get Attributes Associated With Consecutive datetimeDaily High/Low on intraday dataPandas DataFrame and DateTimeIndexFor each day, subtract last time row from the first time row with PythonCreate plots of DataFrames using Python
I need to make an analytical report for a daily pattern. There is a huge dataset.
I need only 2 columns: DateTime and Size. I made DateTime as datetime type and set it as index. That's what I got:
Size
DateTime
2018-10-19 04:14:01.015000+00:00 2
2018-10-19 04:14:01.546000+00:00 1
2018-10-19 04:15:01.290000+00:00 1
2018-10-19 04:15:01.291000+00:00 10
2018-10-19 04:15:01.821000+00:00 1
2018-10-19 04:15:01.821000+00:00 1
2018-10-19 04:15:02.352000+00:00 1
It's only head(7) rows, but there are 1.5 million rows.
I need to make day plot - how size depends on time, but I'm stack in the interval.
How could I set step size for every 5-10 minutes, for example?
python pandas statistics jupyter-notebook analytics
asked Mar 7 at 20:12
SomeOneInterestingSomeOneInteresting
357
add a comment |
I need to make an analytical report for a daily pattern. There is a huge dataset.
I need only 2 columns: DateTime and Size. I made DateTime as datetime type and set it as index. That's what I got:
Size
DateTime
2018-10-19 04:14:01.015000+00:00 2
2018-10-19 04:14:01.546000+00:00 1
2018-10-19 04:15:01.290000+00:00 1
2018-10-19 04:15:01.291000+00:00 10
2018-10-19 04:15:01.821000+00:00 1
2018-10-19 04:15:01.821000+00:00 1
2018-10-19 04:15:02.352000+00:00 1
It's only head(7) rows, but there are 1.5 million rows.
I need to make day plot - how size depends on time, but I'm stack in the interval.
How could I set step size for every 5-10 minutes, for example?
python pandas statistics jupyter-notebook analytics
asked Mar 7 at 20:12
SomeOneInterestingSomeOneInteresting
357
is data regular i mean is every munite same like 12 frame per munite?
– Rifat Alptekin Çetin
Mar 7 at 20:21
Your question is hard to understand. Do you want to group by 5 or 10 minute time intervals? Would you want to average the size of these groups? Sum? Something else?
– pistolpete
Mar 7 at 20:21
Well, I need a plot that could be easily understood that size depends of time
– SomeOneInteresting
Mar 7 at 20:35
add a comment |
I need to make an analytical report for a daily pattern. There is a huge dataset.
I need only 2 columns: DateTime and Size. I made DateTime as datetime type and set it as index. That's what I got:
Size
DateTime
2018-10-19 04:14:01.015000+00:00 2
2018-10-19 04:14:01.546000+00:00 1
2018-10-19 04:15:01.290000+00:00 1
2018-10-19 04:15:01.291000+00:00 10
2018-10-19 04:15:01.821000+00:00 1
2018-10-19 04:15:01.821000+00:00 1
2018-10-19 04:15:02.352000+00:00 1
It's only head(7) rows, but there are 1.5 million rows.
I need to make day plot - how size depends on time, but I'm stack in the interval.
How could I set step size for every 5-10 minutes, for example?
python pandas statistics jupyter-notebook analytics
asked Mar 7 at 20:12
SomeOneInterestingSomeOneInteresting
357
I need to make an analytical report for a daily pattern. There is a huge dataset.
I need only 2 columns: DateTime and Size. I made DateTime as datetime type and set it as index. That's what I got:
Size
DateTime
2018-10-19 04:14:01.015000+00:00 2
2018-10-19 04:14:01.546000+00:00 1
2018-10-19 04:15:01.290000+00:00 1
2018-10-19 04:15:01.291000+00:00 10
2018-10-19 04:15:01.821000+00:00 1
2018-10-19 04:15:01.821000+00:00 1
2018-10-19 04:15:02.352000+00:00 1
It's only head(7) rows, but there are 1.5 million rows.
I need to make day plot - how size depends on time, but I'm stack in the interval.
How could I set step size for every 5-10 minutes, for example?
python pandas statistics jupyter-notebook analytics
python pandas statistics jupyter-notebook analytics
asked Mar 7 at 20:12
SomeOneInterestingSomeOneInteresting
357
asked Mar 7 at 20:12
SomeOneInterestingSomeOneInteresting
357
asked Mar 7 at 20:12
SomeOneInterestingSomeOneInteresting
357
asked Mar 7 at 20:12
SomeOneInterestingSomeOneInteresting
357
asked Mar 7 at 20:12
SomeOneInterestingSomeOneInteresting
357
357
is data regular i mean is every munite same like 12 frame per munite?
– Rifat Alptekin Çetin
Mar 7 at 20:21
Your question is hard to understand. Do you want to group by 5 or 10 minute time intervals? Would you want to average the size of these groups? Sum? Something else?
– pistolpete
Mar 7 at 20:21
Well, I need a plot that could be easily understood that size depends of time
– SomeOneInteresting
Mar 7 at 20:35
add a comment |
is data regular i mean is every munite same like 12 frame per munite?
– Rifat Alptekin Çetin
Mar 7 at 20:21
Your question is hard to understand. Do you want to group by 5 or 10 minute time intervals? Would you want to average the size of these groups? Sum? Something else?
– pistolpete
Mar 7 at 20:21
Well, I need a plot that could be easily understood that size depends of time
– SomeOneInteresting
Mar 7 at 20:35
is data regular i mean is every munite same like 12 frame per munite?
– Rifat Alptekin Çetin
Mar 7 at 20:21
is data regular i mean is every munite same like 12 frame per munite?
– Rifat Alptekin Çetin
Mar 7 at 20:21
Your question is hard to understand. Do you want to group by 5 or 10 minute time intervals? Would you want to average the size of these groups? Sum? Something else?
– pistolpete
Mar 7 at 20:21
Your question is hard to understand. Do you want to group by 5 or 10 minute time intervals? Would you want to average the size of these groups? Sum? Something else?
– pistolpete
Mar 7 at 20:21
Well, I need a plot that could be easily understood that size depends of time
– SomeOneInteresting
Mar 7 at 20:35
Well, I need a plot that could be easily understood that size depends of time
– SomeOneInteresting
Mar 7 at 20:35
add a comment |
0
active
oldest
votes
Your Answer
StackExchange.ifUsing("editor", function ()
StackExchange.using("externalEditor", function ()
StackExchange.using("snippets", function ()
StackExchange.snippets.init();
);
);
, "code-snippets");
StackExchange.ready(function()
var channelOptions =
tags: "".split(" "),
id: "1"
;
initTagRenderer("".split(" "), "".split(" "), channelOptions);
StackExchange.using("externalEditor", function()
// Have to fire editor after snippets, if snippets enabled
if (StackExchange.settings.snippets.snippetsEnabled)
StackExchange.using("snippets", function()
createEditor();
);
else
createEditor();
);
function createEditor()
StackExchange.prepareEditor(
heartbeatType: 'answer',
autoActivateHeartbeat: false,
convertImagesToLinks: true,
noModals: true,
showLowRepImageUploadWarning: true,
reputationToPostImages: 10,
bindNavPrevention: true,
postfix: "",
imageUploader:
brandingHtml: "Powered by u003ca class="icon-imgur-white" href="https://imgur.com/"u003eu003c/au003e",
contentPolicyHtml: "User contributions licensed under u003ca href="https://creativecommons.org/licenses/by-sa/3.0/"u003ecc by-sa 3.0 with attribution requiredu003c/au003e u003ca href="https://stackoverflow.com/legal/content-policy"u003e(content policy)u003c/au003e",
allowUrls: true
,
onDemand: true,
discardSelector: ".discard-answer"
,immediatelyShowMarkdownHelp:true
);
);
Sign up or log in
StackExchange.ready(function ()
StackExchange.helpers.onClickDraftSave('#login-link');
);
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Post as a guest
Required, but never shown
StackExchange.ready(
function ()
StackExchange.openid.initPostLogin('.new-post-login', 'https%3a%2f%2fstackoverflow.com%2fquestions%2f55052081%2fhave-a-huge-data-set-and-i-need-to-analyze-it-financial-tools%23new-answer', 'question_page');
);
Post as a guest
Required, but never shown
0
active
oldest
votes
0
active
oldest
votes
active
oldest
votes
active
oldest
votes
Thanks for contributing an answer to Stack Overflow!
- Please be sure to answer the question. Provide details and share your research!
But avoid …
- Asking for help, clarification, or responding to other answers.
- Making statements based on opinion; back them up with references or personal experience.
To learn more, see our tips on writing great answers.
Sign up or log in
StackExchange.ready(function ()
StackExchange.helpers.onClickDraftSave('#login-link');
);
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Post as a guest
Required, but never shown
StackExchange.ready(
function ()
StackExchange.openid.initPostLogin('.new-post-login', 'https%3a%2f%2fstackoverflow.com%2fquestions%2f55052081%2fhave-a-huge-data-set-and-i-need-to-analyze-it-financial-tools%23new-answer', 'question_page');
);
Post as a guest
Required, but never shown
Sign up or log in
StackExchange.ready(function ()
StackExchange.helpers.onClickDraftSave('#login-link');
);
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Post as a guest
Required, but never shown
Sign up or log in
StackExchange.ready(function ()
StackExchange.helpers.onClickDraftSave('#login-link');
);
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Post as a guest
Required, but never shown
Sign up or log in
StackExchange.ready(function ()
StackExchange.helpers.onClickDraftSave('#login-link');
);
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Post as a guest
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
is data regular i mean is every munite same like 12 frame per munite?
– Rifat Alptekin Çetin
Mar 7 at 20:21
Your question is hard to understand. Do you want to group by 5 or 10 minute time intervals? Would you want to average the size of these groups? Sum? Something else?
– pistolpete
Mar 7 at 20:21
Well, I need a plot that could be easily understood that size depends of time
– SomeOneInteresting
Mar 7 at 20:35