Spring batch parallel processing create steps based on the step1 resultHow to use decider in Spring batch?Spring Batch: how to pass jobParameters to a custom bean?Spring batch - executing one step back OR Re-executing previously executed stepsSpring Batch: How to exit job skipping steps based on conditionHow do I use a date job parameter in Grails spring batch & SQL?Add parameter to job context from tasklet step and use in later steps in Spring BatchSpring Batch define conditional flow java-based using steps in @JobScopeSpring Batch - How to generate parallel steps based on params created in a previous stepHow Does Spring Batch Step Scope Workspring batch step restartSpring Batch Late Binding - Step Scope or Job Scope?
Electoral considerations aside, what are potential benefits, for the US, of policy changes proposed by the tweet recognizing Golan annexation?
Why electric field inside a cavity of a non-conducting sphere not zero?
Why can Carol Danvers change her suit colours in the first place?
Are paving bricks differently sized for sand bedding vs mortar bedding?
Which one is correct as adjective “protruding” or “protruded”?
Did arcade monitors have same pixel aspect ratio as TV sets?
Is it better practice to read straight from sheet music rather than memorize it?
What is Cash Advance APR?
The IT department bottlenecks progress. How should I handle this?
Offered money to buy a house, seller is asking for more to cover gap between their listing and mortgage owed
250 Floor Tower
How do you respond to a colleague from another team when they're wrongly expecting that you'll help them?
Why Shazam when there is already Superman?
Strong empirical falsification of quantum mechanics based on vacuum energy density
What should you do if you miss a job interview (deliberately)?
A social experiment. What is the worst that can happen?
Fear of getting stuck on one programming language / technology that is not used in my country
Aragorn's "guise" in the Orthanc Stone
Is it safe to use olive oil to clean the ear wax?
Is this toilet slogan correct usage of the English language?
Is it possible to have a strip of cold climate in the middle of a planet?
How could a planet have erratic days?
If infinitesimal transformations commute why dont the generators of the Lorentz group commute?
Creepy dinosaur pc game identification
Spring batch parallel processing create steps based on the step1 result
How to use decider in Spring batch?Spring Batch: how to pass jobParameters to a custom bean?Spring batch - executing one step back OR Re-executing previously executed stepsSpring Batch: How to exit job skipping steps based on conditionHow do I use a date job parameter in Grails spring batch & SQL?Add parameter to job context from tasklet step and use in later steps in Spring BatchSpring Batch define conditional flow java-based using steps in @JobScopeSpring Batch - How to generate parallel steps based on params created in a previous stepHow Does Spring Batch Step Scope Workspring batch step restartSpring Batch Late Binding - Step Scope or Job Scope?
I am trying to implement parallel processing using spring boot and spring batch. This batch will be triggered from UI with some required parameters
I need to create steps based on the request parameters, I tried as below,
The rest controller looks,
JobParameters jobParameters = new JobParametersBuilder().addLong("JobID",System.currentTimeMillis())
.addString("fileName", filename)
.addString("buisinessDate", model.getGeneralServiceModel().getBusinessDate())
.addString("source", model.getGeneralServiceModel().getSource())
.toJobParameters();
jobLauncher.run(job, jobParameters);
And the batch config:
Flow masterFlow = (Flow)new FlowBuilder("masterFlow").start(stepOne()).build();
List<Step> steps = new ArrayList<Step>();
for (ConcurrentLinkedQueue date : taskOne.readFile())
steps.add(createStep(date));
return jobs.get("myJob")
.start(masterFlow)
.next(createParallelFlow(steps))
.end()
.build();
The masterFlow reads the job parameters into its variables, and the readFile() gives the list (based on this the steps has to be created), for this the jobParameters are required.
The problem is:
While starting my application itself the readFile() is getting executed. but I need to get it executed when job triggers through RestController since it has the required parameters.
Please help me how can I stop this execution while starting the application.
multithreading spring-boot parallel-processing spring-batch tasklet
add a comment |
I am trying to implement parallel processing using spring boot and spring batch. This batch will be triggered from UI with some required parameters
I need to create steps based on the request parameters, I tried as below,
The rest controller looks,
JobParameters jobParameters = new JobParametersBuilder().addLong("JobID",System.currentTimeMillis())
.addString("fileName", filename)
.addString("buisinessDate", model.getGeneralServiceModel().getBusinessDate())
.addString("source", model.getGeneralServiceModel().getSource())
.toJobParameters();
jobLauncher.run(job, jobParameters);
And the batch config:
Flow masterFlow = (Flow)new FlowBuilder("masterFlow").start(stepOne()).build();
List<Step> steps = new ArrayList<Step>();
for (ConcurrentLinkedQueue date : taskOne.readFile())
steps.add(createStep(date));
return jobs.get("myJob")
.start(masterFlow)
.next(createParallelFlow(steps))
.end()
.build();
The masterFlow reads the job parameters into its variables, and the readFile() gives the list (based on this the steps has to be created), for this the jobParameters are required.
The problem is:
While starting my application itself the readFile() is getting executed. but I need to get it executed when job triggers through RestController since it has the required parameters.
Please help me how can I stop this execution while starting the application.
multithreading spring-boot parallel-processing spring-batch tasklet
It is not clear what you are asking. In the title you saycreate steps based on the step1 result
and in the question you sayI need to create steps based on the request parameters
. Those are different things. Please clarify you question and give an example with real parameters to understand what you are trying to achieve.
– Mahmoud Ben Hassine
Mar 8 at 13:49
I need steps to be created based on step1 result. But in order to process step1 I need request parameters.
– Vijaya
Mar 11 at 5:15
add a comment |
I am trying to implement parallel processing using spring boot and spring batch. This batch will be triggered from UI with some required parameters
I need to create steps based on the request parameters, I tried as below,
The rest controller looks,
JobParameters jobParameters = new JobParametersBuilder().addLong("JobID",System.currentTimeMillis())
.addString("fileName", filename)
.addString("buisinessDate", model.getGeneralServiceModel().getBusinessDate())
.addString("source", model.getGeneralServiceModel().getSource())
.toJobParameters();
jobLauncher.run(job, jobParameters);
And the batch config:
Flow masterFlow = (Flow)new FlowBuilder("masterFlow").start(stepOne()).build();
List<Step> steps = new ArrayList<Step>();
for (ConcurrentLinkedQueue date : taskOne.readFile())
steps.add(createStep(date));
return jobs.get("myJob")
.start(masterFlow)
.next(createParallelFlow(steps))
.end()
.build();
The masterFlow reads the job parameters into its variables, and the readFile() gives the list (based on this the steps has to be created), for this the jobParameters are required.
The problem is:
While starting my application itself the readFile() is getting executed. but I need to get it executed when job triggers through RestController since it has the required parameters.
Please help me how can I stop this execution while starting the application.
multithreading spring-boot parallel-processing spring-batch tasklet
I am trying to implement parallel processing using spring boot and spring batch. This batch will be triggered from UI with some required parameters
I need to create steps based on the request parameters, I tried as below,
The rest controller looks,
JobParameters jobParameters = new JobParametersBuilder().addLong("JobID",System.currentTimeMillis())
.addString("fileName", filename)
.addString("buisinessDate", model.getGeneralServiceModel().getBusinessDate())
.addString("source", model.getGeneralServiceModel().getSource())
.toJobParameters();
jobLauncher.run(job, jobParameters);
And the batch config:
Flow masterFlow = (Flow)new FlowBuilder("masterFlow").start(stepOne()).build();
List<Step> steps = new ArrayList<Step>();
for (ConcurrentLinkedQueue date : taskOne.readFile())
steps.add(createStep(date));
return jobs.get("myJob")
.start(masterFlow)
.next(createParallelFlow(steps))
.end()
.build();
The masterFlow reads the job parameters into its variables, and the readFile() gives the list (based on this the steps has to be created), for this the jobParameters are required.
The problem is:
While starting my application itself the readFile() is getting executed. but I need to get it executed when job triggers through RestController since it has the required parameters.
Please help me how can I stop this execution while starting the application.
multithreading spring-boot parallel-processing spring-batch tasklet
multithreading spring-boot parallel-processing spring-batch tasklet
asked Mar 8 at 4:41
VijayaVijaya
518
518
It is not clear what you are asking. In the title you saycreate steps based on the step1 result
and in the question you sayI need to create steps based on the request parameters
. Those are different things. Please clarify you question and give an example with real parameters to understand what you are trying to achieve.
– Mahmoud Ben Hassine
Mar 8 at 13:49
I need steps to be created based on step1 result. But in order to process step1 I need request parameters.
– Vijaya
Mar 11 at 5:15
add a comment |
It is not clear what you are asking. In the title you saycreate steps based on the step1 result
and in the question you sayI need to create steps based on the request parameters
. Those are different things. Please clarify you question and give an example with real parameters to understand what you are trying to achieve.
– Mahmoud Ben Hassine
Mar 8 at 13:49
I need steps to be created based on step1 result. But in order to process step1 I need request parameters.
– Vijaya
Mar 11 at 5:15
It is not clear what you are asking. In the title you say
create steps based on the step1 result
and in the question you say I need to create steps based on the request parameters
. Those are different things. Please clarify you question and give an example with real parameters to understand what you are trying to achieve.– Mahmoud Ben Hassine
Mar 8 at 13:49
It is not clear what you are asking. In the title you say
create steps based on the step1 result
and in the question you say I need to create steps based on the request parameters
. Those are different things. Please clarify you question and give an example with real parameters to understand what you are trying to achieve.– Mahmoud Ben Hassine
Mar 8 at 13:49
I need steps to be created based on step1 result. But in order to process step1 I need request parameters.
– Vijaya
Mar 11 at 5:15
I need steps to be created based on step1 result. But in order to process step1 I need request parameters.
– Vijaya
Mar 11 at 5:15
add a comment |
1 Answer
1
active
oldest
votes
I need steps to be created based on step1 result
Creating steps is something that you do at configuration time. Step1 result can only be known at runtime. So in order to do this, you would need to access the application context at runtime and register step beans dynamically based on the result of Step1. I am not sure if this is really what you aim to do.
However, if you want to execute (not create, but execute) those steps based on step1's result, then you can use a JobExecutionDecider
. Please refer to Programmatic Flow Decisions for more details and code example. A similar question can be found here: How to use decider in Spring batch?
But in order to process step1 I need request parameters.
I see you have managed to get request parameters and set them as job parameters. What you can do is access those job parameters in your step through the step execution. Here is an example if your step is a simple Tasklet:
class MyTasklet implements Tasklet
@Override
public RepeatStatus execute(StepContribution contribution, ChunkContext chunkContext) throws Exception
Map<String, Object> jobParameters = chunkContext.getStepContext().getJobParameters();
// use job parameters
return RepeatStatus.FINISHED;
If your step is a chunk-oriented tasklet, you can use a StepListener#beforeStep
to get access to the step execution.
Hope this helps.
Okay thanks. I will try your approach.
– Vijaya
Mar 11 at 13:19
add a comment |
Your Answer
StackExchange.ifUsing("editor", function ()
StackExchange.using("externalEditor", function ()
StackExchange.using("snippets", function ()
StackExchange.snippets.init();
);
);
, "code-snippets");
StackExchange.ready(function()
var channelOptions =
tags: "".split(" "),
id: "1"
;
initTagRenderer("".split(" "), "".split(" "), channelOptions);
StackExchange.using("externalEditor", function()
// Have to fire editor after snippets, if snippets enabled
if (StackExchange.settings.snippets.snippetsEnabled)
StackExchange.using("snippets", function()
createEditor();
);
else
createEditor();
);
function createEditor()
StackExchange.prepareEditor(
heartbeatType: 'answer',
autoActivateHeartbeat: false,
convertImagesToLinks: true,
noModals: true,
showLowRepImageUploadWarning: true,
reputationToPostImages: 10,
bindNavPrevention: true,
postfix: "",
imageUploader:
brandingHtml: "Powered by u003ca class="icon-imgur-white" href="https://imgur.com/"u003eu003c/au003e",
contentPolicyHtml: "User contributions licensed under u003ca href="https://creativecommons.org/licenses/by-sa/3.0/"u003ecc by-sa 3.0 with attribution requiredu003c/au003e u003ca href="https://stackoverflow.com/legal/content-policy"u003e(content policy)u003c/au003e",
allowUrls: true
,
onDemand: true,
discardSelector: ".discard-answer"
,immediatelyShowMarkdownHelp:true
);
);
Sign up or log in
StackExchange.ready(function ()
StackExchange.helpers.onClickDraftSave('#login-link');
);
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Post as a guest
Required, but never shown
StackExchange.ready(
function ()
StackExchange.openid.initPostLogin('.new-post-login', 'https%3a%2f%2fstackoverflow.com%2fquestions%2f55056842%2fspring-batch-parallel-processing-create-steps-based-on-the-step1-result%23new-answer', 'question_page');
);
Post as a guest
Required, but never shown
1 Answer
1
active
oldest
votes
1 Answer
1
active
oldest
votes
active
oldest
votes
active
oldest
votes
I need steps to be created based on step1 result
Creating steps is something that you do at configuration time. Step1 result can only be known at runtime. So in order to do this, you would need to access the application context at runtime and register step beans dynamically based on the result of Step1. I am not sure if this is really what you aim to do.
However, if you want to execute (not create, but execute) those steps based on step1's result, then you can use a JobExecutionDecider
. Please refer to Programmatic Flow Decisions for more details and code example. A similar question can be found here: How to use decider in Spring batch?
But in order to process step1 I need request parameters.
I see you have managed to get request parameters and set them as job parameters. What you can do is access those job parameters in your step through the step execution. Here is an example if your step is a simple Tasklet:
class MyTasklet implements Tasklet
@Override
public RepeatStatus execute(StepContribution contribution, ChunkContext chunkContext) throws Exception
Map<String, Object> jobParameters = chunkContext.getStepContext().getJobParameters();
// use job parameters
return RepeatStatus.FINISHED;
If your step is a chunk-oriented tasklet, you can use a StepListener#beforeStep
to get access to the step execution.
Hope this helps.
Okay thanks. I will try your approach.
– Vijaya
Mar 11 at 13:19
add a comment |
I need steps to be created based on step1 result
Creating steps is something that you do at configuration time. Step1 result can only be known at runtime. So in order to do this, you would need to access the application context at runtime and register step beans dynamically based on the result of Step1. I am not sure if this is really what you aim to do.
However, if you want to execute (not create, but execute) those steps based on step1's result, then you can use a JobExecutionDecider
. Please refer to Programmatic Flow Decisions for more details and code example. A similar question can be found here: How to use decider in Spring batch?
But in order to process step1 I need request parameters.
I see you have managed to get request parameters and set them as job parameters. What you can do is access those job parameters in your step through the step execution. Here is an example if your step is a simple Tasklet:
class MyTasklet implements Tasklet
@Override
public RepeatStatus execute(StepContribution contribution, ChunkContext chunkContext) throws Exception
Map<String, Object> jobParameters = chunkContext.getStepContext().getJobParameters();
// use job parameters
return RepeatStatus.FINISHED;
If your step is a chunk-oriented tasklet, you can use a StepListener#beforeStep
to get access to the step execution.
Hope this helps.
Okay thanks. I will try your approach.
– Vijaya
Mar 11 at 13:19
add a comment |
I need steps to be created based on step1 result
Creating steps is something that you do at configuration time. Step1 result can only be known at runtime. So in order to do this, you would need to access the application context at runtime and register step beans dynamically based on the result of Step1. I am not sure if this is really what you aim to do.
However, if you want to execute (not create, but execute) those steps based on step1's result, then you can use a JobExecutionDecider
. Please refer to Programmatic Flow Decisions for more details and code example. A similar question can be found here: How to use decider in Spring batch?
But in order to process step1 I need request parameters.
I see you have managed to get request parameters and set them as job parameters. What you can do is access those job parameters in your step through the step execution. Here is an example if your step is a simple Tasklet:
class MyTasklet implements Tasklet
@Override
public RepeatStatus execute(StepContribution contribution, ChunkContext chunkContext) throws Exception
Map<String, Object> jobParameters = chunkContext.getStepContext().getJobParameters();
// use job parameters
return RepeatStatus.FINISHED;
If your step is a chunk-oriented tasklet, you can use a StepListener#beforeStep
to get access to the step execution.
Hope this helps.
I need steps to be created based on step1 result
Creating steps is something that you do at configuration time. Step1 result can only be known at runtime. So in order to do this, you would need to access the application context at runtime and register step beans dynamically based on the result of Step1. I am not sure if this is really what you aim to do.
However, if you want to execute (not create, but execute) those steps based on step1's result, then you can use a JobExecutionDecider
. Please refer to Programmatic Flow Decisions for more details and code example. A similar question can be found here: How to use decider in Spring batch?
But in order to process step1 I need request parameters.
I see you have managed to get request parameters and set them as job parameters. What you can do is access those job parameters in your step through the step execution. Here is an example if your step is a simple Tasklet:
class MyTasklet implements Tasklet
@Override
public RepeatStatus execute(StepContribution contribution, ChunkContext chunkContext) throws Exception
Map<String, Object> jobParameters = chunkContext.getStepContext().getJobParameters();
// use job parameters
return RepeatStatus.FINISHED;
If your step is a chunk-oriented tasklet, you can use a StepListener#beforeStep
to get access to the step execution.
Hope this helps.
answered Mar 11 at 8:58
Mahmoud Ben HassineMahmoud Ben Hassine
5,3201717
5,3201717
Okay thanks. I will try your approach.
– Vijaya
Mar 11 at 13:19
add a comment |
Okay thanks. I will try your approach.
– Vijaya
Mar 11 at 13:19
Okay thanks. I will try your approach.
– Vijaya
Mar 11 at 13:19
Okay thanks. I will try your approach.
– Vijaya
Mar 11 at 13:19
add a comment |
Thanks for contributing an answer to Stack Overflow!
- Please be sure to answer the question. Provide details and share your research!
But avoid …
- Asking for help, clarification, or responding to other answers.
- Making statements based on opinion; back them up with references or personal experience.
To learn more, see our tips on writing great answers.
Sign up or log in
StackExchange.ready(function ()
StackExchange.helpers.onClickDraftSave('#login-link');
);
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Post as a guest
Required, but never shown
StackExchange.ready(
function ()
StackExchange.openid.initPostLogin('.new-post-login', 'https%3a%2f%2fstackoverflow.com%2fquestions%2f55056842%2fspring-batch-parallel-processing-create-steps-based-on-the-step1-result%23new-answer', 'question_page');
);
Post as a guest
Required, but never shown
Sign up or log in
StackExchange.ready(function ()
StackExchange.helpers.onClickDraftSave('#login-link');
);
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Post as a guest
Required, but never shown
Sign up or log in
StackExchange.ready(function ()
StackExchange.helpers.onClickDraftSave('#login-link');
);
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Post as a guest
Required, but never shown
Sign up or log in
StackExchange.ready(function ()
StackExchange.helpers.onClickDraftSave('#login-link');
);
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Post as a guest
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
It is not clear what you are asking. In the title you say
create steps based on the step1 result
and in the question you sayI need to create steps based on the request parameters
. Those are different things. Please clarify you question and give an example with real parameters to understand what you are trying to achieve.– Mahmoud Ben Hassine
Mar 8 at 13:49
I need steps to be created based on step1 result. But in order to process step1 I need request parameters.
– Vijaya
Mar 11 at 5:15