Nsryjdtyk

How do I apply a function to a list of file paths I have built, and write an output csv in the same path?

read file in a subfolder -> perform a function -> write file in the
subfolder -> go to next subfolder

#opened xml by filename

with open(r'XML_opsReport 100001.xml', encoding = "utf8") as fd:

    Odict_parsedFromFilePath = xmltodict.parse(fd.read()) 



#func called in func below

def activity_to_df_one_day (list_activity_this_day): 

    ib_list = [pd.DataFrame(list_activity_this_day[i], columns=list_activity_this_day[i].keys()).drop("@uom") for i in range(len(list_activity_this_day))]

    return pd.concat(ib_list)



#Processes parsed xml and writes csv 

def activity_to_df_all_days (Odict_parsedFromFilePath, subdir): #writes csv from parsed xml after some processing

    nodes_reports = Odict_parsedFromFilePath['opsReports']['opsReport']

    list_activity = 

    for i in range(len(nodes_reports)):

        try:

            df = activity_to_df_one_day(nodes_reports[i]['activity'])

            list_activity.append(df)



        except KeyError:

            continue

    opsReport = pd.concat(list_activity)

    opsReport['dTimStart'] = pd.to_datetime(opsReport['dTimStart'], infer_datetime_format =True)

    opsReport.sort_values('dTimStart', axis=0, ascending=True, inplace=True, kind='quicksort', na_position='last')

    opsReport.to_csv("subdiropsReport.csv") #write to the subdir









def scanfolder(): #fetches list of file-paths with desired starting name.



    list_files = 



    for path, dirs, files in os.walk(r'C:..xml_objects'): #directory containing several subfolders

        for f in files:

            if f.startswith('XML_opsReport'):

                list_files.append(os.path.join(path, f))

    return list_files



filepaths = scanfolder() #list of file-paths  

Every function works well, the xml processing is good, so I am not sharing the xml structure. There are 100+ paths in filepaths , each a different subdirectory. I want to be able to apply above flow in future as well, where I can get filepaths and perform desired actions. It's important to write the csv file to it's sub directory.

To get the directory that a file is in, you can use:

import os



for root, dirs, files, in os.walk(some_dir):

    for f in files:

        print(root)

        output_file = os.path.join(root, "output_file.csv")

        print(output_file)

Is that what you're looking for?

Output:

somedir

somediroutput_file.csv

See also Python 3 - travel directory tree with limited recursion depth and Find current directory and file's directory.

Was able to solve with os.path.join.

    exceptions_path_list =

    for i in filepaths:

        try:

            with open(i, encoding = "utf8") as fd:

                doc = xmltodict.parse(fd.read())

                activity_to_df_all_days (doc, i)

        except ValueError:

            exceptions_path_list.append(os.path.dirname(i))

            continue



    def activity_to_df_all_days (Odict_parsedFromFilePath, filepath):

        ...

        ...

        ...    

        opsReport.to_csv(os.path.join(os.path.dirname(filepath), "opsReport.csv"))