Start containers in parallelFinding the index of an item given a list containing it in PythonDoes Python have a string 'contains' substring method?How to list containers in DockerHow to remove old Docker containersCopying files from Docker container to hostCopying files from host to Docker containerFrom inside of a Docker container, how do I connect to the localhost of the machine?Parallel test processes within docker containerImplement on-demand docker container start-upAutomatically Generate Pods in Kubernetes

How could a planet have erratic days?

Can someone explain how this makes sense electrically?

Is the U.S. Code copyrighted by the Government?

Why does the Sun have different day lengths, but not the gas giants?

Count the occurrence of each unique word in the file

Electoral considerations aside, what are potential benefits, for the US, of policy changes proposed by the tweet recognizing Golan annexation?

Aragorn's "guise" in the Orthanc Stone

On a tidally locked planet, would time be quantized?

Why do compilers behave differently when static_cast(ing) a function to void*?

GraphicsGrid with a Label for each Column and Row

Yosemite Fire Rings - What to Expect?

Where does the bonus feat in the cleric starting package come from?

Loading commands from file

If a character has darkvision, can they see through an area of nonmagical darkness filled with lightly obscuring gas?

Does an advisor owe his/her student anything? Will an advisor keep a PhD student only out of pity?

Question about the proof of Second Isomorphism Theorem

Did arcade monitors have same pixel aspect ratio as TV sets?

How much character growth crosses the line into breaking the character

Removing files under particular conditions (number of files, file age)

What prevents the use of a multi-segment ILS for non-straight approaches?

Why should universal income be universal?

What is this called? Old film camera viewer?

Non-trope happy ending?

Creature in Shazam mid-credits scene?



Start containers in parallel


Finding the index of an item given a list containing it in PythonDoes Python have a string 'contains' substring method?How to list containers in DockerHow to remove old Docker containersCopying files from Docker container to hostCopying files from host to Docker containerFrom inside of a Docker container, how do I connect to the localhost of the machine?Parallel test processes within docker containerImplement on-demand docker container start-upAutomatically Generate Pods in Kubernetes













0















Is it possible to start Docker containers in parallel using docker-py without having Kubernetes or swarm being involved?



I tried Gevent to asynchronously run 2 functions, each for creating and starting a container. I spawned let's say 10 gevent threads each for creating and starting 10 containers in parallel.
From the logging timestamps, creating containers seems to be done in parallel. But starting containers seems to be done in synchronously. The timestamps show 10 gevent threads start a container one after another.



I kind of feel like it is impossible since docker daemon probably needs to maintain something that involves locking, such as network configurations. I googled for days about this but I couldn't find any good explanation why it is impossible. I saw similar questions on stackoverflow but all of them remain unanswered, or were not properly answered to the point.










share|improve this question






















  • Example from the docs: client.containers.run("bfirsh/reticulate-splines", detach=True) Now run as many as you want and your resources allow.

    – Klaus D.
    Mar 8 at 5:08











  • My question is how to run multiple containers in parallel. The reason I’m asking here is running the example code like you wrote in parallel doesn’t actually run containers in parallel.

    – Pyson
    Mar 9 at 6:15












  • This is how you ran one. I leave it to you to figure out how to run two or more based on this.

    – Klaus D.
    Mar 9 at 6:17











  • I don’t think you got what I am asking...

    – Pyson
    Mar 9 at 6:24
















0















Is it possible to start Docker containers in parallel using docker-py without having Kubernetes or swarm being involved?



I tried Gevent to asynchronously run 2 functions, each for creating and starting a container. I spawned let's say 10 gevent threads each for creating and starting 10 containers in parallel.
From the logging timestamps, creating containers seems to be done in parallel. But starting containers seems to be done in synchronously. The timestamps show 10 gevent threads start a container one after another.



I kind of feel like it is impossible since docker daemon probably needs to maintain something that involves locking, such as network configurations. I googled for days about this but I couldn't find any good explanation why it is impossible. I saw similar questions on stackoverflow but all of them remain unanswered, or were not properly answered to the point.










share|improve this question






















  • Example from the docs: client.containers.run("bfirsh/reticulate-splines", detach=True) Now run as many as you want and your resources allow.

    – Klaus D.
    Mar 8 at 5:08











  • My question is how to run multiple containers in parallel. The reason I’m asking here is running the example code like you wrote in parallel doesn’t actually run containers in parallel.

    – Pyson
    Mar 9 at 6:15












  • This is how you ran one. I leave it to you to figure out how to run two or more based on this.

    – Klaus D.
    Mar 9 at 6:17











  • I don’t think you got what I am asking...

    – Pyson
    Mar 9 at 6:24














0












0








0








Is it possible to start Docker containers in parallel using docker-py without having Kubernetes or swarm being involved?



I tried Gevent to asynchronously run 2 functions, each for creating and starting a container. I spawned let's say 10 gevent threads each for creating and starting 10 containers in parallel.
From the logging timestamps, creating containers seems to be done in parallel. But starting containers seems to be done in synchronously. The timestamps show 10 gevent threads start a container one after another.



I kind of feel like it is impossible since docker daemon probably needs to maintain something that involves locking, such as network configurations. I googled for days about this but I couldn't find any good explanation why it is impossible. I saw similar questions on stackoverflow but all of them remain unanswered, or were not properly answered to the point.










share|improve this question














Is it possible to start Docker containers in parallel using docker-py without having Kubernetes or swarm being involved?



I tried Gevent to asynchronously run 2 functions, each for creating and starting a container. I spawned let's say 10 gevent threads each for creating and starting 10 containers in parallel.
From the logging timestamps, creating containers seems to be done in parallel. But starting containers seems to be done in synchronously. The timestamps show 10 gevent threads start a container one after another.



I kind of feel like it is impossible since docker daemon probably needs to maintain something that involves locking, such as network configurations. I googled for days about this but I couldn't find any good explanation why it is impossible. I saw similar questions on stackoverflow but all of them remain unanswered, or were not properly answered to the point.







python docker gevent dockerpy






share|improve this question













share|improve this question











share|improve this question




share|improve this question










asked Mar 8 at 5:03









PysonPyson

265




265












  • Example from the docs: client.containers.run("bfirsh/reticulate-splines", detach=True) Now run as many as you want and your resources allow.

    – Klaus D.
    Mar 8 at 5:08











  • My question is how to run multiple containers in parallel. The reason I’m asking here is running the example code like you wrote in parallel doesn’t actually run containers in parallel.

    – Pyson
    Mar 9 at 6:15












  • This is how you ran one. I leave it to you to figure out how to run two or more based on this.

    – Klaus D.
    Mar 9 at 6:17











  • I don’t think you got what I am asking...

    – Pyson
    Mar 9 at 6:24


















  • Example from the docs: client.containers.run("bfirsh/reticulate-splines", detach=True) Now run as many as you want and your resources allow.

    – Klaus D.
    Mar 8 at 5:08











  • My question is how to run multiple containers in parallel. The reason I’m asking here is running the example code like you wrote in parallel doesn’t actually run containers in parallel.

    – Pyson
    Mar 9 at 6:15












  • This is how you ran one. I leave it to you to figure out how to run two or more based on this.

    – Klaus D.
    Mar 9 at 6:17











  • I don’t think you got what I am asking...

    – Pyson
    Mar 9 at 6:24

















Example from the docs: client.containers.run("bfirsh/reticulate-splines", detach=True) Now run as many as you want and your resources allow.

– Klaus D.
Mar 8 at 5:08





Example from the docs: client.containers.run("bfirsh/reticulate-splines", detach=True) Now run as many as you want and your resources allow.

– Klaus D.
Mar 8 at 5:08













My question is how to run multiple containers in parallel. The reason I’m asking here is running the example code like you wrote in parallel doesn’t actually run containers in parallel.

– Pyson
Mar 9 at 6:15






My question is how to run multiple containers in parallel. The reason I’m asking here is running the example code like you wrote in parallel doesn’t actually run containers in parallel.

– Pyson
Mar 9 at 6:15














This is how you ran one. I leave it to you to figure out how to run two or more based on this.

– Klaus D.
Mar 9 at 6:17





This is how you ran one. I leave it to you to figure out how to run two or more based on this.

– Klaus D.
Mar 9 at 6:17













I don’t think you got what I am asking...

– Pyson
Mar 9 at 6:24






I don’t think you got what I am asking...

– Pyson
Mar 9 at 6:24













0






active

oldest

votes











Your Answer






StackExchange.ifUsing("editor", function ()
StackExchange.using("externalEditor", function ()
StackExchange.using("snippets", function ()
StackExchange.snippets.init();
);
);
, "code-snippets");

StackExchange.ready(function()
var channelOptions =
tags: "".split(" "),
id: "1"
;
initTagRenderer("".split(" "), "".split(" "), channelOptions);

StackExchange.using("externalEditor", function()
// Have to fire editor after snippets, if snippets enabled
if (StackExchange.settings.snippets.snippetsEnabled)
StackExchange.using("snippets", function()
createEditor();
);

else
createEditor();

);

function createEditor()
StackExchange.prepareEditor(
heartbeatType: 'answer',
autoActivateHeartbeat: false,
convertImagesToLinks: true,
noModals: true,
showLowRepImageUploadWarning: true,
reputationToPostImages: 10,
bindNavPrevention: true,
postfix: "",
imageUploader:
brandingHtml: "Powered by u003ca class="icon-imgur-white" href="https://imgur.com/"u003eu003c/au003e",
contentPolicyHtml: "User contributions licensed under u003ca href="https://creativecommons.org/licenses/by-sa/3.0/"u003ecc by-sa 3.0 with attribution requiredu003c/au003e u003ca href="https://stackoverflow.com/legal/content-policy"u003e(content policy)u003c/au003e",
allowUrls: true
,
onDemand: true,
discardSelector: ".discard-answer"
,immediatelyShowMarkdownHelp:true
);



);













draft saved

draft discarded


















StackExchange.ready(
function ()
StackExchange.openid.initPostLogin('.new-post-login', 'https%3a%2f%2fstackoverflow.com%2fquestions%2f55057012%2fstart-containers-in-parallel%23new-answer', 'question_page');

);

Post as a guest















Required, but never shown

























0






active

oldest

votes








0






active

oldest

votes









active

oldest

votes






active

oldest

votes















draft saved

draft discarded
















































Thanks for contributing an answer to Stack Overflow!


  • Please be sure to answer the question. Provide details and share your research!

But avoid


  • Asking for help, clarification, or responding to other answers.

  • Making statements based on opinion; back them up with references or personal experience.

To learn more, see our tips on writing great answers.




draft saved


draft discarded














StackExchange.ready(
function ()
StackExchange.openid.initPostLogin('.new-post-login', 'https%3a%2f%2fstackoverflow.com%2fquestions%2f55057012%2fstart-containers-in-parallel%23new-answer', 'question_page');

);

Post as a guest















Required, but never shown





















































Required, but never shown














Required, but never shown












Required, but never shown







Required, but never shown

































Required, but never shown














Required, but never shown












Required, but never shown







Required, but never shown







Popular posts from this blog

Thal And Out Agency railway station See also References External links Navigation menuOfficial Web Site of Pakistan RailwaysArchivedOfficial Web Site of Pakistan Railwayseeexpanding ite

Understanding generators in Python2019 Community Moderator ElectionGenerator function not working pythonFor loop not executing two timesGenerators - Printing generated valuesWhy can a python generator only be used once?What exactly do generators do?What does the “yield” keyword do?What does “list comprehension” mean? How does it work and how can I use it?sklearn Kfold acces single fold instead of for loopIs a generator the callable? Which is the generator?Apply Border To Range Of Cells Using OpenpyxlCalling an external command in PythonWhat are metaclasses in Python?What is the difference between @staticmethod and @classmethod?Finding the index of an item given a list containing it in PythonDifference between append vs. extend list methods in PythonHow can I safely create a nested directory in Python?Does Python have a ternary conditional operator?Understanding slice notationUnderstanding Python super() with __init__() methodsDoes Python have a string 'contains' substring method?

How can I change the color of pagination dots of UIPageControl?How to change UIPageControl dotsIs there a way to change page indicator dots colorCustomize dot with image of UIPageControl at index 0 of UIPageControlNo visible @interface for 'NSObject<PageControlDelegate>' declares the selector 'pageControlPageDidChange:'How to change the color of pagination dots in UIPageControl with a different color per pagepagecontrol indicator custom image instead of DefaultChanging the colour of UIPageControl dots in MonoTouchpagecontrol selectable page visibility color?Alternative way to load ViewControllers on a UIPageControlHow to set only layer.border-color for UIpage control dots in swiftHow can I develop for iPhone using a Windows development machine?How to change the name of an iOS app?UITableView - change section header coloruipagecontrol indicator(dot)issueCustom UIPageControl dots color not changingchange the interspace between UIPageControl dotsHow to change Status Bar text color in iOSHow can I change image tintColor in iOS and WatchKitUIPageControl dots with larger space in between each dotsHow to change the color of pagination dots in UIPageControl with a different color per page