Ggtcf

I'm calling rsync with popen and the output is not printing out continuously in my python script for my web application as it does in just normal linux. I'm trying to copy over all the files on one directory to another (massive copy). I want to use the progress numbers received from the output changing to eventually create/update a progress-bar I have in my web application. All I want is the total progress of the overall copy so I would use --info=progress2 in my rsync command.
I also tried :

while True:
 line = self.proc.stdout.readline()
 if line != '':
 # the real code does filtering here
 print("test:", line.rstrip())
 else:
 break

but that waited till the end to just print test: b''
I think the issues is either with the while loop extracting data or how I print it to my console using a different class too.

There is not much information of using this --info=progress2
since it's a relatively new update.

Here's my code.

import subprocess
import logging
import sys
import os
import replicator.dfp.rep_utils as ru


class SyncProcessor(object):
 def __init__(self, src, dest):
 self.src = src
 self.dest = dest
 self.proc = None
 self.callback = None
 log_file = "sync--.log".format(self.src, self.dest)
 self.sync_logger = ru.setup_logger(__file__, log_file, level=logging.DEBUG)

 def add_sync_callback(self, cb):
 self.callback = cb

 def run(self):
 print("Syncing Drive "+ str(self.src.driveNum) + " to Drive " + str(self.dest.driveNum))
 rsync_cmd = "sudo rsync -ah --info=progress2 --delete --stats /media/drive/ /media/drive".format(self.src.driveNum, self.dest.driveNum)
 self.proc = subprocess.Popen(rsync_cmd, shell=True, stdout=subprocess.PIPE, stderr=subprocess.PIPE)

 while self.proc.poll() is None:
 output = self.proc.stdout.readline()
 if output == '':
 break
 if output:
 print("OUTPUT DECODE: " + output.decode("utf-8")
 #self.sync_logger.info(output.decode())
 self.callback.update(output.decode())
 print("<< Finished Syncing >>")
 #self.sync_logger.debug("<< Finished Syncing >>")
 rc = self.proc.poll()
 #self.sync_logger.debug("Return code: ".format(rc))
 os.system("sync")
 return rc

 def communicate(self):
 return self.proc.communicate()

class Progress(object):
 """Callback to report progress of a SyncProcessor"""
 def __init__(self, src, dest, out=sys.stdout):
 self.src = src
 self.dest = dest
 self.out = out

 def update(self, data):
 line = "From Progress(-) -> "
 self.out.write(line.format(self.src, self.dest, data))

So what I realized was the whole changing of percent from 0-100% was treated as one line since it was broken up by r instead of n

self.proc.stdout.readline()

therefore this line only activates after process reaches 100%

I switched it to self.proc.stdout.readline(80)
which worked as it printed out every 80 characters giving me updates on the precentage. However since length of line changes throughout I'm looking for a better way to do this