Python waiting for operation to complete before continuing

Submitted 3 years, 7 months ago

Ticket #160

Views 222

Language/Framework Python

Priority Medium

Status Closed

I'm writing data to a CSV file, then once that's done, I copy the file to another directory.

This is all in a loop, so when the second iteration starts it reads the data from the file that was copied.

The problem is that the file is still being copied while the second iteration starts and this causes obvious issues.

How would I wait for the whole function in the loop to be complete before the second iterations starts? it should be able to go on with any amount of iterations.

for rule in substring_rules:
    substring(rule)

the function:

def substring(rule, remove_rows=[]):        
            writer = csv.writer(open("%s%s" % (DZINE_DIR, f), "wb"))
            from_column = rule.from_column
            to_column = rule.to_column
            reader = csv.reader(open("%s%s" % (OUTPUT_DIR, f)))
            headers = reader.next()
            index = 0
            from_column_index = None
            for head in headers:
                if head == from_column:
                    from_column_index = index
                index += 1

            if to_column not in headers:
                headers.append(to_column)

            writer.writerow(headers)

            row_index = 0
            for row in reader:
                if rule.get_rule_type_display() == "substring":
                    try:
                        string = rule.string.split(",")
                        new_value = string[0] + row[from_column_index] + string[1]
                        if from_column == to_column:
                            row[from_column_index] = new_value
                        else:
                            row.append(new_value)
                    except Exception, e:
                        print e

                if row_index not in remove_rows:
                    writer.writerow(row)
                row_index += 1
            shutil.copyfile("%s%s" % (DZINE_DIR,f), "%s%s" % (OUTPUT_DIR, f))

Submitted on Sep 13, 20

Anonymous

add a comment

1 Answer

Verified

You can try this step "to make Python wait".

from random import random
import threading
import time

result = None

def background_calculation():
    # here goes some long calculation
    time.sleep(random() * 5 * 60)

    # when the calculation is done, the result is stored in a global variable
    global result
    result = 42

def main():
    thread = threading.Thread(target=background_calculation)
    thread.start()

    # wait here for the result to be available before continuing
    while result is None:
        pass

    print('The result is', result)

if __name__ == '__main__':
    main()

Submitted 3 years, 6 months ago