Keras ImageDataGenerator sample_weight with data augmentationHow to determine amount of augmented images in Keras?Fit generator and data augmentation in kerasKeras AttributeError: 'list' object has no attribute 'ndim'LSTM with Keras: Input 'ref' of 'Assign' Op requires l-value inputWhat are the arguments in function fit of keras?Error while doing reshapeIOError: [Errno 2] No such file or directory when training Keras modelHow many images are generated by keras fit_generator?Keras Image data augmentationNeural Network classification

Is this apparent Class Action settlement a spam message?

What Brexit proposals are on the table in the indicative votes on the 27th of March 2019?

What is the difference between "behavior" and "behaviour"?

Unreliable Magic - Is it worth it?

Do sorcerers' Subtle Spells require a skill check to be unseen?

How easy is it to start Magic from scratch?

How does Loki do this?

Roman Numeral Treatment of Suspensions

How did Doctor Strange see the winning outcome in Avengers: Infinity War?

Avoiding estate tax by giving multiple gifts

India just shot down a satellite from the ground. At what altitude range is the resulting debris field?

What can we do to stop prior company from asking us questions?

Class Action - which options I have?

Is exact Kanji stroke length important?

Detecting if an element is found inside a container

How does it work when somebody invests in my business?

Where does the Z80 processor start executing from?

What is paid subscription needed for in Mortal Kombat 11?

Is `x >> pure y` equivalent to `liftM (const y) x`

How can a function with a hole (removable discontinuity) equal a function with no hole?

Why escape if the_content isnt?

Proof of work - lottery approach

What is the best translation for "slot" in the context of multiplayer video games?

Increase performance creating Mandelbrot set in python



Keras ImageDataGenerator sample_weight with data augmentation


How to determine amount of augmented images in Keras?Fit generator and data augmentation in kerasKeras AttributeError: 'list' object has no attribute 'ndim'LSTM with Keras: Input 'ref' of 'Assign' Op requires l-value inputWhat are the arguments in function fit of keras?Error while doing reshapeIOError: [Errno 2] No such file or directory when training Keras modelHow many images are generated by keras fit_generator?Keras Image data augmentationNeural Network classification













1















I have a question about the use of the sample_weight parameter in the context of data augmentation in Keras with the ImageDataGenerator. Let's say I have a series of simple images with just one class of objects. So, for each image, I will have a corresponding mask with pixels = 0 for the background and 1 for where the object is labeled.



However, this dataset is unbalanced because a significant amount of these images are empty, which mean with masks just containing 0.
If I understood well, the 'sample_weight' parameter of the flow method of ImageDataGenerator is here to put the focus on the the samples of my dataset that I find more interesting, i.e. where my object is present.



My question is: what is the concrete influence of this sample_weight parameter on the training of my model. Does it influence the data augmentation? If I use the 'validation_split' parameter, does it influence the way validation sets are generated?



Here is the part of my code my question refers to:





data_gen_args = dict(rotation_range=90,
width_shift_range=0.4,
height_shift_range=0.4,
zoom_range=0.4,
horizontal_flip=True,
fill_mode='reflect',
rescale=1. / 255,
validation_split=0.2,
data_format='channels_last'
)

image_datagen = ImageDataGenerator(**data_gen_args)


imf = image_datagen.flow(
x=stacked_images_channel,
y=stacked_masks_channel,
batch_size=batch_size,
shuffle=False,
seed=seed,subset='training',
sample_weight = sample_weight,
save_to_dir = 'traindir',
save_prefix = 'train_'
)

valf = image_datagen.flow(
x=stacked_images_channel,
y=stacked_masks_channel,
batch_size=batch_size,
shuffle=False,
seed=seed,subset='validation',
sample_weight = sample_weight,
save_to_dir = 'valdir',
save_prefix = 'val_'
)

STEP_SIZE_TRAIN=imf.n//imf.batch_size
STEP_SIZE_VALID=valf.n//valf.batch_size

model = unet.UNet2(numberOfClasses, imshape, '', learningRate, depth=4)

history = model.fit_generator(generator=imf,
steps_per_epoch=STEP_SIZE_TRAIN,
epochs=epochs,
validation_data=valf,
validation_steps=STEP_SIZE_VALID,
verbose=2
)


Thank you in advance for your attention.










share|improve this question
























  • Hi again! It seems that my question does not inspire a lot of people. Sorry to ask again but is there really no one out there who understands well this sample_weight feature? I thought there would be someone from the Keras team itself or at least a well-experienced user. I would really like to know how I could use this for my problem. Thank you in advance for your attention.

    – Maxclac
    yesterday
















1















I have a question about the use of the sample_weight parameter in the context of data augmentation in Keras with the ImageDataGenerator. Let's say I have a series of simple images with just one class of objects. So, for each image, I will have a corresponding mask with pixels = 0 for the background and 1 for where the object is labeled.



However, this dataset is unbalanced because a significant amount of these images are empty, which mean with masks just containing 0.
If I understood well, the 'sample_weight' parameter of the flow method of ImageDataGenerator is here to put the focus on the the samples of my dataset that I find more interesting, i.e. where my object is present.



My question is: what is the concrete influence of this sample_weight parameter on the training of my model. Does it influence the data augmentation? If I use the 'validation_split' parameter, does it influence the way validation sets are generated?



Here is the part of my code my question refers to:





data_gen_args = dict(rotation_range=90,
width_shift_range=0.4,
height_shift_range=0.4,
zoom_range=0.4,
horizontal_flip=True,
fill_mode='reflect',
rescale=1. / 255,
validation_split=0.2,
data_format='channels_last'
)

image_datagen = ImageDataGenerator(**data_gen_args)


imf = image_datagen.flow(
x=stacked_images_channel,
y=stacked_masks_channel,
batch_size=batch_size,
shuffle=False,
seed=seed,subset='training',
sample_weight = sample_weight,
save_to_dir = 'traindir',
save_prefix = 'train_'
)

valf = image_datagen.flow(
x=stacked_images_channel,
y=stacked_masks_channel,
batch_size=batch_size,
shuffle=False,
seed=seed,subset='validation',
sample_weight = sample_weight,
save_to_dir = 'valdir',
save_prefix = 'val_'
)

STEP_SIZE_TRAIN=imf.n//imf.batch_size
STEP_SIZE_VALID=valf.n//valf.batch_size

model = unet.UNet2(numberOfClasses, imshape, '', learningRate, depth=4)

history = model.fit_generator(generator=imf,
steps_per_epoch=STEP_SIZE_TRAIN,
epochs=epochs,
validation_data=valf,
validation_steps=STEP_SIZE_VALID,
verbose=2
)


Thank you in advance for your attention.










share|improve this question
























  • Hi again! It seems that my question does not inspire a lot of people. Sorry to ask again but is there really no one out there who understands well this sample_weight feature? I thought there would be someone from the Keras team itself or at least a well-experienced user. I would really like to know how I could use this for my problem. Thank you in advance for your attention.

    – Maxclac
    yesterday














1












1








1








I have a question about the use of the sample_weight parameter in the context of data augmentation in Keras with the ImageDataGenerator. Let's say I have a series of simple images with just one class of objects. So, for each image, I will have a corresponding mask with pixels = 0 for the background and 1 for where the object is labeled.



However, this dataset is unbalanced because a significant amount of these images are empty, which mean with masks just containing 0.
If I understood well, the 'sample_weight' parameter of the flow method of ImageDataGenerator is here to put the focus on the the samples of my dataset that I find more interesting, i.e. where my object is present.



My question is: what is the concrete influence of this sample_weight parameter on the training of my model. Does it influence the data augmentation? If I use the 'validation_split' parameter, does it influence the way validation sets are generated?



Here is the part of my code my question refers to:





data_gen_args = dict(rotation_range=90,
width_shift_range=0.4,
height_shift_range=0.4,
zoom_range=0.4,
horizontal_flip=True,
fill_mode='reflect',
rescale=1. / 255,
validation_split=0.2,
data_format='channels_last'
)

image_datagen = ImageDataGenerator(**data_gen_args)


imf = image_datagen.flow(
x=stacked_images_channel,
y=stacked_masks_channel,
batch_size=batch_size,
shuffle=False,
seed=seed,subset='training',
sample_weight = sample_weight,
save_to_dir = 'traindir',
save_prefix = 'train_'
)

valf = image_datagen.flow(
x=stacked_images_channel,
y=stacked_masks_channel,
batch_size=batch_size,
shuffle=False,
seed=seed,subset='validation',
sample_weight = sample_weight,
save_to_dir = 'valdir',
save_prefix = 'val_'
)

STEP_SIZE_TRAIN=imf.n//imf.batch_size
STEP_SIZE_VALID=valf.n//valf.batch_size

model = unet.UNet2(numberOfClasses, imshape, '', learningRate, depth=4)

history = model.fit_generator(generator=imf,
steps_per_epoch=STEP_SIZE_TRAIN,
epochs=epochs,
validation_data=valf,
validation_steps=STEP_SIZE_VALID,
verbose=2
)


Thank you in advance for your attention.










share|improve this question
















I have a question about the use of the sample_weight parameter in the context of data augmentation in Keras with the ImageDataGenerator. Let's say I have a series of simple images with just one class of objects. So, for each image, I will have a corresponding mask with pixels = 0 for the background and 1 for where the object is labeled.



However, this dataset is unbalanced because a significant amount of these images are empty, which mean with masks just containing 0.
If I understood well, the 'sample_weight' parameter of the flow method of ImageDataGenerator is here to put the focus on the the samples of my dataset that I find more interesting, i.e. where my object is present.



My question is: what is the concrete influence of this sample_weight parameter on the training of my model. Does it influence the data augmentation? If I use the 'validation_split' parameter, does it influence the way validation sets are generated?



Here is the part of my code my question refers to:





data_gen_args = dict(rotation_range=90,
width_shift_range=0.4,
height_shift_range=0.4,
zoom_range=0.4,
horizontal_flip=True,
fill_mode='reflect',
rescale=1. / 255,
validation_split=0.2,
data_format='channels_last'
)

image_datagen = ImageDataGenerator(**data_gen_args)


imf = image_datagen.flow(
x=stacked_images_channel,
y=stacked_masks_channel,
batch_size=batch_size,
shuffle=False,
seed=seed,subset='training',
sample_weight = sample_weight,
save_to_dir = 'traindir',
save_prefix = 'train_'
)

valf = image_datagen.flow(
x=stacked_images_channel,
y=stacked_masks_channel,
batch_size=batch_size,
shuffle=False,
seed=seed,subset='validation',
sample_weight = sample_weight,
save_to_dir = 'valdir',
save_prefix = 'val_'
)

STEP_SIZE_TRAIN=imf.n//imf.batch_size
STEP_SIZE_VALID=valf.n//valf.batch_size

model = unet.UNet2(numberOfClasses, imshape, '', learningRate, depth=4)

history = model.fit_generator(generator=imf,
steps_per_epoch=STEP_SIZE_TRAIN,
epochs=epochs,
validation_data=valf,
validation_steps=STEP_SIZE_VALID,
verbose=2
)


Thank you in advance for your attention.







keras






share|improve this question















share|improve this question













share|improve this question




share|improve this question








edited Mar 8 at 14:06









Ioannis Nasios

3,75831036




3,75831036










asked Mar 8 at 10:58









MaxclacMaxclac

41




41












  • Hi again! It seems that my question does not inspire a lot of people. Sorry to ask again but is there really no one out there who understands well this sample_weight feature? I thought there would be someone from the Keras team itself or at least a well-experienced user. I would really like to know how I could use this for my problem. Thank you in advance for your attention.

    – Maxclac
    yesterday


















  • Hi again! It seems that my question does not inspire a lot of people. Sorry to ask again but is there really no one out there who understands well this sample_weight feature? I thought there would be someone from the Keras team itself or at least a well-experienced user. I would really like to know how I could use this for my problem. Thank you in advance for your attention.

    – Maxclac
    yesterday

















Hi again! It seems that my question does not inspire a lot of people. Sorry to ask again but is there really no one out there who understands well this sample_weight feature? I thought there would be someone from the Keras team itself or at least a well-experienced user. I would really like to know how I could use this for my problem. Thank you in advance for your attention.

– Maxclac
yesterday






Hi again! It seems that my question does not inspire a lot of people. Sorry to ask again but is there really no one out there who understands well this sample_weight feature? I thought there would be someone from the Keras team itself or at least a well-experienced user. I would really like to know how I could use this for my problem. Thank you in advance for your attention.

– Maxclac
yesterday













0






active

oldest

votes











Your Answer






StackExchange.ifUsing("editor", function ()
StackExchange.using("externalEditor", function ()
StackExchange.using("snippets", function ()
StackExchange.snippets.init();
);
);
, "code-snippets");

StackExchange.ready(function()
var channelOptions =
tags: "".split(" "),
id: "1"
;
initTagRenderer("".split(" "), "".split(" "), channelOptions);

StackExchange.using("externalEditor", function()
// Have to fire editor after snippets, if snippets enabled
if (StackExchange.settings.snippets.snippetsEnabled)
StackExchange.using("snippets", function()
createEditor();
);

else
createEditor();

);

function createEditor()
StackExchange.prepareEditor(
heartbeatType: 'answer',
autoActivateHeartbeat: false,
convertImagesToLinks: true,
noModals: true,
showLowRepImageUploadWarning: true,
reputationToPostImages: 10,
bindNavPrevention: true,
postfix: "",
imageUploader:
brandingHtml: "Powered by u003ca class="icon-imgur-white" href="https://imgur.com/"u003eu003c/au003e",
contentPolicyHtml: "User contributions licensed under u003ca href="https://creativecommons.org/licenses/by-sa/3.0/"u003ecc by-sa 3.0 with attribution requiredu003c/au003e u003ca href="https://stackoverflow.com/legal/content-policy"u003e(content policy)u003c/au003e",
allowUrls: true
,
onDemand: true,
discardSelector: ".discard-answer"
,immediatelyShowMarkdownHelp:true
);



);













draft saved

draft discarded


















StackExchange.ready(
function ()
StackExchange.openid.initPostLogin('.new-post-login', 'https%3a%2f%2fstackoverflow.com%2fquestions%2f55061774%2fkeras-imagedatagenerator-sample-weight-with-data-augmentation%23new-answer', 'question_page');

);

Post as a guest















Required, but never shown

























0






active

oldest

votes








0






active

oldest

votes









active

oldest

votes






active

oldest

votes















draft saved

draft discarded
















































Thanks for contributing an answer to Stack Overflow!


  • Please be sure to answer the question. Provide details and share your research!

But avoid …


  • Asking for help, clarification, or responding to other answers.

  • Making statements based on opinion; back them up with references or personal experience.

To learn more, see our tips on writing great answers.




draft saved


draft discarded














StackExchange.ready(
function ()
StackExchange.openid.initPostLogin('.new-post-login', 'https%3a%2f%2fstackoverflow.com%2fquestions%2f55061774%2fkeras-imagedatagenerator-sample-weight-with-data-augmentation%23new-answer', 'question_page');

);

Post as a guest















Required, but never shown





















































Required, but never shown














Required, but never shown












Required, but never shown







Required, but never shown

































Required, but never shown














Required, but never shown












Required, but never shown







Required, but never shown







Popular posts from this blog

Thal And Out Agency railway station See also References External links Navigation menuOfficial Web Site of Pakistan RailwaysArchivedOfficial Web Site of Pakistan Railwayseeexpanding ite

Posting a File and Associated Data to a RESTful WebService preferably as JSON2019 Community Moderator ElectionREST URI convention - Singular or plural name of resource while creating itUploading both data and files in one form using Ajax?How do I upload a file with metadata using a REST web service?REST API - file (ie images) processing - best practicesSpring: JSON data and file in the same requestREST API Put large dataPosting a file and JSON data to Spring rest serviceRESTful Web Service Upload/Download Large Data With JSONREST API Design sending JSON data and a file to the api in same requestPost JSONArray to REST servicePUT vs. POST in RESTParsing values from a JSON file?JavaScript/jQuery to download file via POST with JSON dataHow do I upload a file with metadata using a REST web service?How to POST JSON data with Curl from Terminal/Commandline to Test Spring REST?Separate REST JSON API server and client?what's the correct way to send a file from REST web service to client?How do I write JSON data to a file?What is “406-Not Acceptable Response” in HTTP?REST API - file (ie images) processing - best practices

How can I change the color of pagination dots of UIPageControl?How to change UIPageControl dotsIs there a way to change page indicator dots colorCustomize dot with image of UIPageControl at index 0 of UIPageControlNo visible @interface for 'NSObject<PageControlDelegate>' declares the selector 'pageControlPageDidChange:'How to change the color of pagination dots in UIPageControl with a different color per pagepagecontrol indicator custom image instead of DefaultChanging the colour of UIPageControl dots in MonoTouchpagecontrol selectable page visibility color?Alternative way to load ViewControllers on a UIPageControlHow to set only layer.border-color for UIpage control dots in swiftHow can I develop for iPhone using a Windows development machine?How to change the name of an iOS app?UITableView - change section header coloruipagecontrol indicator(dot)issueCustom UIPageControl dots color not changingchange the interspace between UIPageControl dotsHow to change Status Bar text color in iOSHow can I change image tintColor in iOS and WatchKitUIPageControl dots with larger space in between each dotsHow to change the color of pagination dots in UIPageControl with a different color per page