Spring batch parallel processing create steps based on the step1 resultHow to use decider in Spring batch?Spring Batch: how to pass jobParameters to a custom bean?Spring batch - executing one step back OR Re-executing previously executed stepsSpring Batch: How to exit job skipping steps based on conditionHow do I use a date job parameter in Grails spring batch & SQL?Add parameter to job context from tasklet step and use in later steps in Spring BatchSpring Batch define conditional flow java-based using steps in @JobScopeSpring Batch - How to generate parallel steps based on params created in a previous stepHow Does Spring Batch Step Scope Workspring batch step restartSpring Batch Late Binding - Step Scope or Job Scope?

Electoral considerations aside, what are potential benefits, for the US, of policy changes proposed by the tweet recognizing Golan annexation?

Why electric field inside a cavity of a non-conducting sphere not zero?

Why can Carol Danvers change her suit colours in the first place?

Are paving bricks differently sized for sand bedding vs mortar bedding?

Which one is correct as adjective “protruding” or “protruded”?

Did arcade monitors have same pixel aspect ratio as TV sets?

Is it better practice to read straight from sheet music rather than memorize it?

What is Cash Advance APR?

The IT department bottlenecks progress. How should I handle this?

Offered money to buy a house, seller is asking for more to cover gap between their listing and mortgage owed

250 Floor Tower

How do you respond to a colleague from another team when they're wrongly expecting that you'll help them?

Why Shazam when there is already Superman?

Strong empirical falsification of quantum mechanics based on vacuum energy density

What should you do if you miss a job interview (deliberately)?

A social experiment. What is the worst that can happen?

Fear of getting stuck on one programming language / technology that is not used in my country

Aragorn's "guise" in the Orthanc Stone

Is it safe to use olive oil to clean the ear wax?

Is this toilet slogan correct usage of the English language?

Is it possible to have a strip of cold climate in the middle of a planet?

How could a planet have erratic days?

If infinitesimal transformations commute why dont the generators of the Lorentz group commute?

Creepy dinosaur pc game identification



Spring batch parallel processing create steps based on the step1 result


How to use decider in Spring batch?Spring Batch: how to pass jobParameters to a custom bean?Spring batch - executing one step back OR Re-executing previously executed stepsSpring Batch: How to exit job skipping steps based on conditionHow do I use a date job parameter in Grails spring batch & SQL?Add parameter to job context from tasklet step and use in later steps in Spring BatchSpring Batch define conditional flow java-based using steps in @JobScopeSpring Batch - How to generate parallel steps based on params created in a previous stepHow Does Spring Batch Step Scope Workspring batch step restartSpring Batch Late Binding - Step Scope or Job Scope?













0















I am trying to implement parallel processing using spring boot and spring batch. This batch will be triggered from UI with some required parameters



I need to create steps based on the request parameters, I tried as below,



The rest controller looks,



JobParameters jobParameters = new JobParametersBuilder().addLong("JobID",System.currentTimeMillis())
.addString("fileName", filename)
.addString("buisinessDate", model.getGeneralServiceModel().getBusinessDate())
.addString("source", model.getGeneralServiceModel().getSource())
.toJobParameters();
jobLauncher.run(job, jobParameters);


And the batch config:



Flow masterFlow = (Flow)new FlowBuilder("masterFlow").start(stepOne()).build();

List<Step> steps = new ArrayList<Step>();
for (ConcurrentLinkedQueue date : taskOne.readFile())
steps.add(createStep(date));


return jobs.get("myJob")
.start(masterFlow)
.next(createParallelFlow(steps))
.end()
.build();


The masterFlow reads the job parameters into its variables, and the readFile() gives the list (based on this the steps has to be created), for this the jobParameters are required.



The problem is:



While starting my application itself the readFile() is getting executed. but I need to get it executed when job triggers through RestController since it has the required parameters.



Please help me how can I stop this execution while starting the application.










share|improve this question






















  • It is not clear what you are asking. In the title you say create steps based on the step1 result and in the question you say I need to create steps based on the request parameters. Those are different things. Please clarify you question and give an example with real parameters to understand what you are trying to achieve.

    – Mahmoud Ben Hassine
    Mar 8 at 13:49












  • I need steps to be created based on step1 result. But in order to process step1 I need request parameters.

    – Vijaya
    Mar 11 at 5:15















0















I am trying to implement parallel processing using spring boot and spring batch. This batch will be triggered from UI with some required parameters



I need to create steps based on the request parameters, I tried as below,



The rest controller looks,



JobParameters jobParameters = new JobParametersBuilder().addLong("JobID",System.currentTimeMillis())
.addString("fileName", filename)
.addString("buisinessDate", model.getGeneralServiceModel().getBusinessDate())
.addString("source", model.getGeneralServiceModel().getSource())
.toJobParameters();
jobLauncher.run(job, jobParameters);


And the batch config:



Flow masterFlow = (Flow)new FlowBuilder("masterFlow").start(stepOne()).build();

List<Step> steps = new ArrayList<Step>();
for (ConcurrentLinkedQueue date : taskOne.readFile())
steps.add(createStep(date));


return jobs.get("myJob")
.start(masterFlow)
.next(createParallelFlow(steps))
.end()
.build();


The masterFlow reads the job parameters into its variables, and the readFile() gives the list (based on this the steps has to be created), for this the jobParameters are required.



The problem is:



While starting my application itself the readFile() is getting executed. but I need to get it executed when job triggers through RestController since it has the required parameters.



Please help me how can I stop this execution while starting the application.










share|improve this question






















  • It is not clear what you are asking. In the title you say create steps based on the step1 result and in the question you say I need to create steps based on the request parameters. Those are different things. Please clarify you question and give an example with real parameters to understand what you are trying to achieve.

    – Mahmoud Ben Hassine
    Mar 8 at 13:49












  • I need steps to be created based on step1 result. But in order to process step1 I need request parameters.

    – Vijaya
    Mar 11 at 5:15













0












0








0








I am trying to implement parallel processing using spring boot and spring batch. This batch will be triggered from UI with some required parameters



I need to create steps based on the request parameters, I tried as below,



The rest controller looks,



JobParameters jobParameters = new JobParametersBuilder().addLong("JobID",System.currentTimeMillis())
.addString("fileName", filename)
.addString("buisinessDate", model.getGeneralServiceModel().getBusinessDate())
.addString("source", model.getGeneralServiceModel().getSource())
.toJobParameters();
jobLauncher.run(job, jobParameters);


And the batch config:



Flow masterFlow = (Flow)new FlowBuilder("masterFlow").start(stepOne()).build();

List<Step> steps = new ArrayList<Step>();
for (ConcurrentLinkedQueue date : taskOne.readFile())
steps.add(createStep(date));


return jobs.get("myJob")
.start(masterFlow)
.next(createParallelFlow(steps))
.end()
.build();


The masterFlow reads the job parameters into its variables, and the readFile() gives the list (based on this the steps has to be created), for this the jobParameters are required.



The problem is:



While starting my application itself the readFile() is getting executed. but I need to get it executed when job triggers through RestController since it has the required parameters.



Please help me how can I stop this execution while starting the application.










share|improve this question














I am trying to implement parallel processing using spring boot and spring batch. This batch will be triggered from UI with some required parameters



I need to create steps based on the request parameters, I tried as below,



The rest controller looks,



JobParameters jobParameters = new JobParametersBuilder().addLong("JobID",System.currentTimeMillis())
.addString("fileName", filename)
.addString("buisinessDate", model.getGeneralServiceModel().getBusinessDate())
.addString("source", model.getGeneralServiceModel().getSource())
.toJobParameters();
jobLauncher.run(job, jobParameters);


And the batch config:



Flow masterFlow = (Flow)new FlowBuilder("masterFlow").start(stepOne()).build();

List<Step> steps = new ArrayList<Step>();
for (ConcurrentLinkedQueue date : taskOne.readFile())
steps.add(createStep(date));


return jobs.get("myJob")
.start(masterFlow)
.next(createParallelFlow(steps))
.end()
.build();


The masterFlow reads the job parameters into its variables, and the readFile() gives the list (based on this the steps has to be created), for this the jobParameters are required.



The problem is:



While starting my application itself the readFile() is getting executed. but I need to get it executed when job triggers through RestController since it has the required parameters.



Please help me how can I stop this execution while starting the application.







multithreading spring-boot parallel-processing spring-batch tasklet






share|improve this question













share|improve this question











share|improve this question




share|improve this question










asked Mar 8 at 4:41









VijayaVijaya

518




518












  • It is not clear what you are asking. In the title you say create steps based on the step1 result and in the question you say I need to create steps based on the request parameters. Those are different things. Please clarify you question and give an example with real parameters to understand what you are trying to achieve.

    – Mahmoud Ben Hassine
    Mar 8 at 13:49












  • I need steps to be created based on step1 result. But in order to process step1 I need request parameters.

    – Vijaya
    Mar 11 at 5:15

















  • It is not clear what you are asking. In the title you say create steps based on the step1 result and in the question you say I need to create steps based on the request parameters. Those are different things. Please clarify you question and give an example with real parameters to understand what you are trying to achieve.

    – Mahmoud Ben Hassine
    Mar 8 at 13:49












  • I need steps to be created based on step1 result. But in order to process step1 I need request parameters.

    – Vijaya
    Mar 11 at 5:15
















It is not clear what you are asking. In the title you say create steps based on the step1 result and in the question you say I need to create steps based on the request parameters. Those are different things. Please clarify you question and give an example with real parameters to understand what you are trying to achieve.

– Mahmoud Ben Hassine
Mar 8 at 13:49






It is not clear what you are asking. In the title you say create steps based on the step1 result and in the question you say I need to create steps based on the request parameters. Those are different things. Please clarify you question and give an example with real parameters to understand what you are trying to achieve.

– Mahmoud Ben Hassine
Mar 8 at 13:49














I need steps to be created based on step1 result. But in order to process step1 I need request parameters.

– Vijaya
Mar 11 at 5:15





I need steps to be created based on step1 result. But in order to process step1 I need request parameters.

– Vijaya
Mar 11 at 5:15












1 Answer
1






active

oldest

votes


















0















I need steps to be created based on step1 result




Creating steps is something that you do at configuration time. Step1 result can only be known at runtime. So in order to do this, you would need to access the application context at runtime and register step beans dynamically based on the result of Step1. I am not sure if this is really what you aim to do.



However, if you want to execute (not create, but execute) those steps based on step1's result, then you can use a JobExecutionDecider. Please refer to Programmatic Flow Decisions for more details and code example. A similar question can be found here: How to use decider in Spring batch?




But in order to process step1 I need request parameters.




I see you have managed to get request parameters and set them as job parameters. What you can do is access those job parameters in your step through the step execution. Here is an example if your step is a simple Tasklet:



class MyTasklet implements Tasklet 
@Override
public RepeatStatus execute(StepContribution contribution, ChunkContext chunkContext) throws Exception
Map<String, Object> jobParameters = chunkContext.getStepContext().getJobParameters();
// use job parameters
return RepeatStatus.FINISHED;




If your step is a chunk-oriented tasklet, you can use a StepListener#beforeStep to get access to the step execution.



Hope this helps.






share|improve this answer























  • Okay thanks. I will try your approach.

    – Vijaya
    Mar 11 at 13:19










Your Answer






StackExchange.ifUsing("editor", function ()
StackExchange.using("externalEditor", function ()
StackExchange.using("snippets", function ()
StackExchange.snippets.init();
);
);
, "code-snippets");

StackExchange.ready(function()
var channelOptions =
tags: "".split(" "),
id: "1"
;
initTagRenderer("".split(" "), "".split(" "), channelOptions);

StackExchange.using("externalEditor", function()
// Have to fire editor after snippets, if snippets enabled
if (StackExchange.settings.snippets.snippetsEnabled)
StackExchange.using("snippets", function()
createEditor();
);

else
createEditor();

);

function createEditor()
StackExchange.prepareEditor(
heartbeatType: 'answer',
autoActivateHeartbeat: false,
convertImagesToLinks: true,
noModals: true,
showLowRepImageUploadWarning: true,
reputationToPostImages: 10,
bindNavPrevention: true,
postfix: "",
imageUploader:
brandingHtml: "Powered by u003ca class="icon-imgur-white" href="https://imgur.com/"u003eu003c/au003e",
contentPolicyHtml: "User contributions licensed under u003ca href="https://creativecommons.org/licenses/by-sa/3.0/"u003ecc by-sa 3.0 with attribution requiredu003c/au003e u003ca href="https://stackoverflow.com/legal/content-policy"u003e(content policy)u003c/au003e",
allowUrls: true
,
onDemand: true,
discardSelector: ".discard-answer"
,immediatelyShowMarkdownHelp:true
);



);













draft saved

draft discarded


















StackExchange.ready(
function ()
StackExchange.openid.initPostLogin('.new-post-login', 'https%3a%2f%2fstackoverflow.com%2fquestions%2f55056842%2fspring-batch-parallel-processing-create-steps-based-on-the-step1-result%23new-answer', 'question_page');

);

Post as a guest















Required, but never shown

























1 Answer
1






active

oldest

votes








1 Answer
1






active

oldest

votes









active

oldest

votes






active

oldest

votes









0















I need steps to be created based on step1 result




Creating steps is something that you do at configuration time. Step1 result can only be known at runtime. So in order to do this, you would need to access the application context at runtime and register step beans dynamically based on the result of Step1. I am not sure if this is really what you aim to do.



However, if you want to execute (not create, but execute) those steps based on step1's result, then you can use a JobExecutionDecider. Please refer to Programmatic Flow Decisions for more details and code example. A similar question can be found here: How to use decider in Spring batch?




But in order to process step1 I need request parameters.




I see you have managed to get request parameters and set them as job parameters. What you can do is access those job parameters in your step through the step execution. Here is an example if your step is a simple Tasklet:



class MyTasklet implements Tasklet 
@Override
public RepeatStatus execute(StepContribution contribution, ChunkContext chunkContext) throws Exception
Map<String, Object> jobParameters = chunkContext.getStepContext().getJobParameters();
// use job parameters
return RepeatStatus.FINISHED;




If your step is a chunk-oriented tasklet, you can use a StepListener#beforeStep to get access to the step execution.



Hope this helps.






share|improve this answer























  • Okay thanks. I will try your approach.

    – Vijaya
    Mar 11 at 13:19















0















I need steps to be created based on step1 result




Creating steps is something that you do at configuration time. Step1 result can only be known at runtime. So in order to do this, you would need to access the application context at runtime and register step beans dynamically based on the result of Step1. I am not sure if this is really what you aim to do.



However, if you want to execute (not create, but execute) those steps based on step1's result, then you can use a JobExecutionDecider. Please refer to Programmatic Flow Decisions for more details and code example. A similar question can be found here: How to use decider in Spring batch?




But in order to process step1 I need request parameters.




I see you have managed to get request parameters and set them as job parameters. What you can do is access those job parameters in your step through the step execution. Here is an example if your step is a simple Tasklet:



class MyTasklet implements Tasklet 
@Override
public RepeatStatus execute(StepContribution contribution, ChunkContext chunkContext) throws Exception
Map<String, Object> jobParameters = chunkContext.getStepContext().getJobParameters();
// use job parameters
return RepeatStatus.FINISHED;




If your step is a chunk-oriented tasklet, you can use a StepListener#beforeStep to get access to the step execution.



Hope this helps.






share|improve this answer























  • Okay thanks. I will try your approach.

    – Vijaya
    Mar 11 at 13:19













0












0








0








I need steps to be created based on step1 result




Creating steps is something that you do at configuration time. Step1 result can only be known at runtime. So in order to do this, you would need to access the application context at runtime and register step beans dynamically based on the result of Step1. I am not sure if this is really what you aim to do.



However, if you want to execute (not create, but execute) those steps based on step1's result, then you can use a JobExecutionDecider. Please refer to Programmatic Flow Decisions for more details and code example. A similar question can be found here: How to use decider in Spring batch?




But in order to process step1 I need request parameters.




I see you have managed to get request parameters and set them as job parameters. What you can do is access those job parameters in your step through the step execution. Here is an example if your step is a simple Tasklet:



class MyTasklet implements Tasklet 
@Override
public RepeatStatus execute(StepContribution contribution, ChunkContext chunkContext) throws Exception
Map<String, Object> jobParameters = chunkContext.getStepContext().getJobParameters();
// use job parameters
return RepeatStatus.FINISHED;




If your step is a chunk-oriented tasklet, you can use a StepListener#beforeStep to get access to the step execution.



Hope this helps.






share|improve this answer














I need steps to be created based on step1 result




Creating steps is something that you do at configuration time. Step1 result can only be known at runtime. So in order to do this, you would need to access the application context at runtime and register step beans dynamically based on the result of Step1. I am not sure if this is really what you aim to do.



However, if you want to execute (not create, but execute) those steps based on step1's result, then you can use a JobExecutionDecider. Please refer to Programmatic Flow Decisions for more details and code example. A similar question can be found here: How to use decider in Spring batch?




But in order to process step1 I need request parameters.




I see you have managed to get request parameters and set them as job parameters. What you can do is access those job parameters in your step through the step execution. Here is an example if your step is a simple Tasklet:



class MyTasklet implements Tasklet 
@Override
public RepeatStatus execute(StepContribution contribution, ChunkContext chunkContext) throws Exception
Map<String, Object> jobParameters = chunkContext.getStepContext().getJobParameters();
// use job parameters
return RepeatStatus.FINISHED;




If your step is a chunk-oriented tasklet, you can use a StepListener#beforeStep to get access to the step execution.



Hope this helps.







share|improve this answer












share|improve this answer



share|improve this answer










answered Mar 11 at 8:58









Mahmoud Ben HassineMahmoud Ben Hassine

5,3201717




5,3201717












  • Okay thanks. I will try your approach.

    – Vijaya
    Mar 11 at 13:19

















  • Okay thanks. I will try your approach.

    – Vijaya
    Mar 11 at 13:19
















Okay thanks. I will try your approach.

– Vijaya
Mar 11 at 13:19





Okay thanks. I will try your approach.

– Vijaya
Mar 11 at 13:19



















draft saved

draft discarded
















































Thanks for contributing an answer to Stack Overflow!


  • Please be sure to answer the question. Provide details and share your research!

But avoid


  • Asking for help, clarification, or responding to other answers.

  • Making statements based on opinion; back them up with references or personal experience.

To learn more, see our tips on writing great answers.




draft saved


draft discarded














StackExchange.ready(
function ()
StackExchange.openid.initPostLogin('.new-post-login', 'https%3a%2f%2fstackoverflow.com%2fquestions%2f55056842%2fspring-batch-parallel-processing-create-steps-based-on-the-step1-result%23new-answer', 'question_page');

);

Post as a guest















Required, but never shown





















































Required, but never shown














Required, but never shown












Required, but never shown







Required, but never shown

































Required, but never shown














Required, but never shown












Required, but never shown







Required, but never shown







Popular posts from this blog

Identity Server 4 is not redirecting to Angular app after login2019 Community Moderator ElectionIdentity Server 4 and dockerIdentityserver implicit flow unauthorized_clientIdentityServer Hybrid Flow - Access Token is null after user successful loginIdentity Server to MVC client : Page Redirect After loginLogin with Steam OpenId(oidc-client-js)Identity Server 4+.NET Core 2.0 + IdentityIdentityServer4 post-login redirect not working in Edge browserCall to IdentityServer4 generates System.NullReferenceException: Object reference not set to an instance of an objectIdentityServer4 without HTTPS not workingHow to get Authorization code from identity server without login form

2005 Ahvaz unrest Contents Background Causes Casualties Aftermath See also References Navigation menue"At Least 10 Are Killed by Bombs in Iran""Iran"Archived"Arab-Iranians in Iran to make April 15 'Day of Fury'"State of Mind, State of Order: Reactions to Ethnic Unrest in the Islamic Republic of Iran.10.1111/j.1754-9469.2008.00028.x"Iran hangs Arab separatists"Iran Overview from ArchivedConstitution of the Islamic Republic of Iran"Tehran puzzled by forged 'riots' letter""Iran and its minorities: Down in the second class""Iran: Handling Of Ahvaz Unrest Could End With Televised Confessions""Bombings Rock Iran Ahead of Election""Five die in Iran ethnic clashes""Iran: Need for restraint as anniversary of unrest in Khuzestan approaches"Archived"Iranian Sunni protesters killed in clashes with security forces"Archived

Can't initialize raids on a new ASUS Prime B360M-A motherboard2019 Community Moderator ElectionSimilar to RAID config yet more like mirroring solution?Can't get motherboard serial numberWhy does the BIOS entry point start with a WBINVD instruction?UEFI performance Asus Maximus V Extreme