Ggtcf

I have a set of files that are read in line by line. I would like have the last line of every file to have the name of the file next to it. This is the code that accomplishes the reading in the file part but I don't know how to get the filenames to show up:

import glob

a = []

def convert_txt_to_dataframe(path):
 for files in glob.glob(path + "./*manual.txt"):
 for x in open(files):
 a.append(x)

So this accomplishes importing all the text files line by line, so now I want every the last line of every file to have an accompanying filename next to it

I want it to look something like:

 Hello Goodbye
0 Thank you for being a loyal customer. MyDocuments/TextFile1
1 Thank you for being a horrible customer. MyDocuments/TextFile1
2 Thank you for being a nice customer. MyDocuments/TextFile3

So I'm assuming you are taking a list of files and those columns you mentioned [0,1,2] are referring to the last lines of each file in your list. With that in mind, I would try a simpler approach instead of a dataframe. And even if you have to use the dataframe for other reasons, perhaps you can convert to text as your last step and try this:

Example File ("ExampleText2"):
I love coffee
I love creamer
I love coffee and creamer
I have a rash..

Code:

last = []
with open('exampleText2.txt', 'r') as f:
 last = f.readlines()[-1] + " other FileName"

Output:

last
'I have a rash.. other FileName'

readlines() will return a list of all the lines in your file, so you could try calling the -1 to pull the last line, then add to it.

I'm assuming that the number of lines is more than or equals the number of files.

import glob

words = ['Thank you for being a loyal customer.',
 'Thank you for being a horrible customer.', 
 'Thank you for being a nice customer.'] 

def convert(path):
 a = []
 z = 0
 for files in glob.glob(path + "/*.txt"):
 temp = [words[z],files]
 a.append(temp)
 z += 1
 print (a) 

convert(your_path)

The question is ill-defined, but assuming the OP wants the result shown in the DataFrame example (i.e. not just the last line is somehow decorated with the filename, but all lines are), here is a way to achieve that. For this example, we just have two files: file1.txt contains two lines: 'a' and 'b', file2.txt contains one line: 'c'.

We write a file reader that returns a list of lists: each sublist contains the filename and a line.

import glob

def get_file(filename):
 with open(filename) as f:
 return [[filename, line.rstrip('n')] for line in f]

Try it:

m = map(get_file, glob.glob('file*.txt'))
list(m)

Out[]:
[[['file2.txt', 'c']], [['file1.txt', 'a'], ['file1.txt', 'b']]]

Let us flatten these lists to get one two-dimensional array. Also, it is probably nicer to get a result where the files are sorted alphabetically.

def flatten(m):
 return [k for sublist in m for k in sublist]

m = map(get_file, sorted(glob.glob('file*.txt')))
flatten(m)

Out[]:
[['file1.txt', 'a'], ['file1.txt', 'b'], ['file2.txt', 'c']]

Now, it sometimes helps to have the line number (say if we are going to put that data in a DataFrame and do further sorting and analytics). Our reader becomes:

def get_file(filename):
 with open(filename) as f:
 return [[filename, lineno, line.rstrip('n')] for lineno, line in enumerate(f, start=1)]

m = map(get_file, sorted(glob.glob('file*.txt')))
out = pd.DataFrame(flatten(m), columns=['filename', 'lineno', 'line'])
out

Out[]:
 filename lineno line
0 file1.txt 1 a
1 file1.txt 2 b
2 file2.txt 1 c

Notice that the map above lends itself nicely to a multi-threaded reading if we do have a large number of files:

from concurrent.futures import ThreadPoolExecutor

with ThreadPoolExecutor(max_workers=4) as pool:
 m = pool.map(get_file, sorted(glob.glob('file*.txt')))
 out = pd.DataFrame(flatten(m), columns=['filename', 'lineno', 'line'])
out

Out[]:
 filename lineno line
0 file1.txt 1 a
1 file1.txt 2 b
2 file2.txt 1 c