How to use MYSQL/PostgreSQL in multi thread python program

Submitted 2 years, 10 months ago
Ticket #431
Views 1216
Language/Framework MYSQL
Priority High
Status Closed

I have a requirement to get the data from MYSQL table, The table has date column.

Based on date filter condition i need to do the multi threading and insert into other tables.

For example, 

Source MYSQL Table(Order)

I need to select * from Order where last_date >= '?'(this date should be passed via from mult threading loops)

From this output ,load/insert into another table lets say(Order1)

Please help me with some sample code.

Submitted on May 04, 21

Our team is working on your ticket, we will try to provide solution as soon as possible, thanks! - Jaanvi 2 years, 10 months ago
add a comment

1 Answer


How about this?

import psycopg2
import random
import time
from concurrent.futures import ThreadPoolExecutor, as_completed
from psycopg2.pool import ThreadedConnectionPool
DSN = "postgresql://user:pass@localhost/database"
tcp = ThreadedConnectionPool(1, 10, DSN)

def write_sim_to_db(all_ids2):
    conn = tcp.getconn()
    c = conn.cursor()
    c.execute("""SELECT * FROM TABLE A WHERE DATA_FILTER= %s""", ("test_data",))

if __name__ == "__main__":
    start_time = time.time()
    all_ids1 = list(n for n in range(10))
    all_ids2_list = list(n for n in range(10))
    for i in range(len(all_ids1)):
        with ThreadPoolExecutor(max_workers=10) as pool:
            results = [pool.submit(write_sim_to_db, i) for i in all_ids2_list]
    elapsed_time = time.time() - start_time


Submitted 2 years, 9 months ago

Wanna Post Ad?
Reach a large group of audience by posting an ad about your business here. For more details
Drop us a Message

Latest Blogs