A complete guide to connecting Python to MySQL and PostgreSQL databases

In Python back-end development, dealing with relational-database is an unavoidable core link. Whether you are a newbie or a developer who wants to sort out best practices, this article will use "from basic connection to production optimization" to help you master the key skills of connecting Python to MySQL and PostgreSQL. We will include a large amount of runnable code so that you can practice while reading.

1. Environment preparation and dependency installation

1.1 Select drivers and tools

For MySQL and PostgreSQL, the mainstream drivers are as follows:

# MySQL 驱动(二选一即可)
pip install PyMySQL                    # 纯 Python 实现,跨平台零配置
pip install mysql-connector-python    # Oracle 官方驱动,支持原生认证

# PostgreSQL 驱动
pip install psycopg2-binary           # 预编译版本,免去编译烦恼

# 通用工具(强烈推荐)
pip install SQLAlchemy                # 明星级 ORM,提供连接池与统一接口
pip install DBUtils                   # 纯连接池工具,适合原生驱动场景
pip install python-dotenv             # 敏感信息与代码分离的安全实践

1.2 Use.envSeparate sensitive information

Never hardcode your database password in your code! use.envFile management connection parameters, coordinationpython-dotenvAnalysis is both safe and convenient.

# .env 文件示例
MYSQL_HOST=localhost
MYSQL_PORT=3306
MYSQL_USER=app_user
MYSQL_PASSWORD=your_strong_password
MYSQL_DATABASE=test_db

PG_HOST=localhost
PG_PORT=5432
PG_USER=app_user
PG_PASSWORD=your_strong_password
PG_DATABASE=test_db

Then write a unified configuration reading module:

# config.py
from dotenv import load_dotenv
import os

load_dotenv()  # 自动读取 .env 文件并注入环境变量

DB_CONFIG = {
    "mysql": {
        "host": os.getenv("MYSQL_HOST"),
        "port": int(os.getenv("MYSQL_PORT", 3306)),
        "user": os.getenv("MYSQL_USER"),
        "password": os.getenv("MYSQL_PASSWORD"),
        "database": os.getenv("MYSQL_DATABASE"),
        "charset": "utf8mb4",
        "autocommit": False,          # 显式控制事务
    },
    "postgresql": {
        "host": os.getenv("PG_HOST"),
        "port": int(os.getenv("PG_PORT", 5432)),
        "user": os.getenv("PG_USER"),
        "password": os.getenv("PG_PASSWORD"),
        "database": os.getenv("PG_DATABASE"),
    }
}

This has two advantages: first, the password is not uploaded to Git, and second, it only needs to be modified when switching environments..envdocument.


2. Basic operations of native driver

First use native PyMySQL and psycopg2 to go through the connection, query, and insertion processes. It will be clearer before introducing ORM after understanding the underlying mechanism.

2.1 Universal connection context manager

Native connections need to be closed and rolled back manually, and are encapsulated with Python's context manager to ensure safety at every step.

import logging
from contextlib import contextmanager

logging.basicConfig(level=logging.INFO)

# MySQL 上下文
def mysql_conn_context(config):
    import pymysql
    conn = None
    try:
        conn = pymysql.connect(**config)
        yield conn
        conn.commit()
    except Exception as e:
        if conn:
            conn.rollback()
        logging.error(f"MySQL 操作失败: {e}")
        raise
    finally:
        if conn:
            conn.close()

# PostgreSQL 上下文
def pg_conn_context(config):
    import psycopg2
    conn = None
    try:
        conn = psycopg2.connect(**config)
        yield conn
        conn.commit()
    except Exception as e:
        if conn:
            conn.rollback()
        logging.error(f"PostgreSQL 操作失败: {e}")
        raise
    finally:
        if conn:
            conn.close()

2.2 MySQL CRUD practice

from config import DB_CONFIG

def create_mysql_user(name, email, age):
    """创建用户,并返回自增主键"""
    with mysql_conn_context(DB_CONFIG["mysql"]) as conn:
        cursor = conn.cursor()
        # 注意:参数化查询必须使用 %s 占位符
        sql = "INSERT INTO users (name, email, age) VALUES (%s, %s, %s)"
        cursor.execute(sql, (name, email, age))
        return cursor.lastrowid

def get_mysql_users(limit=10):
    """查询用户列表,结果以字典形式返回"""
    with mysql_conn_context(DB_CONFIG["mysql"]) as conn:
        # 指定 DictCursor 让结果更容易使用
        cursor = conn.cursor(pymysql.cursors.DictCursor)
        cursor.execute("SELECT * FROM users LIMIT %s", (limit,))
        return cursor.fetchall()

2.3 PostgreSQL-specific skills: RETURNING and dictionary parameters

PostgreSQL provides many practical extensions, such asRETURNINGClause and named parameter placeholders, allowing you to write one less query.

from config import DB_CONFIG

def upsert_pg_user(data):
    """UPSERT 操作:插入,冲突则更新(PostgreSQL 独有)"""
    with pg_conn_context(DB_CONFIG["postgresql"]) as conn:
        cursor = conn.cursor()
        sql = """
            INSERT INTO users (name, email, age)
            VALUES (%(name)s, %(email)s, %(age)s)
            ON CONFLICT (email)                -- 要求 email 列有唯一索引
            DO UPDATE SET age = EXCLUDED.age
            RETURNING id, name, email
        """
        cursor.execute(sql, data)
        return cursor.fetchone()   # 直接拿到插入/更新后的行

Used here%(name)sin the form of directly passed into the dictionarydata, much better readability than sequential placeholders.


3. Connection pool: the core of performance improvement

A native database connection must go through TCP handshake, authentication and other processes every time, and the overhead is on the order of tens of milliseconds. For high-concurrency scenarios, using a connection pool to reuse connections can improve performance by more than 10 times.

3.1 MySQL connection pool (DBUtils)

useDBUtils.PooledDBAdd connection pooling capabilities to PyMySQL:

from DBUtils.PooledDB import PooledDB
import pymysql

# 全局初始化连接池(程序启动时执行一次)
mysql_pool = PooledDB(
    creator=pymysql,           # 使用 PyMySQL 创建连接
    maxconnections=20,         # 最大允许连接数
    mincached=2,               # 池中初始空闲连接
    maxcached=5,               # 池中最大空闲连接
    **DB_CONFIG["mysql"]
)

# 封装为上下文管理器
def mysql_pool_context():
    conn = None
    try:
        conn = mysql_pool.connection()  # 从池中获取连接
        yield conn
        conn.commit()
    except Exception as e:
        if conn:
            conn.rollback()
        logging.error(f"MySQL 连接池操作失败: {e}")
        raise
    finally:
        if conn:
            conn.close()   # close 是归还连接,而非物理断开

The usage is exactly the same as before, just replace the context manager and enjoy connection reuse.

3.2 PostgreSQL connection pool (SQLAlchemy out-of-the-box)

If you have already started using SQLAlchemy, its built-in connection pool function requires no additional configuration:

from sqlalchemy import create_engine
from sqlalchemy.orm import sessionmaker

# 构建 PostgreSQL 引擎,连接池默认启用
pg_engine = create_engine(
    "postgresql+psycopg2://{user}:{password}@{host}:{port}/{database}".format(
        **DB_CONFIG["postgresql"]
    ),
    pool_size=10,           # 连接池保持的连接数
    max_overflow=10,        # 临时允许超出的连接数
    pool_pre_ping=True      # 每次取出连接前先探测是否有效
)

PgSession = sessionmaker(bind=pg_engine)

4. SQLAlchemy ORM: Make database operations more Pythonic

ORM (Object Relational Mapping) maps database tables into Python classes. Writing additions, deletions, modifications and queries is like operating ordinary objects, while automatically handling parameterization, transactions and cross-library compatibility.

4.1 Define data model

from sqlalchemy.ext.declarative import declarative_base
from sqlalchemy import Column, Integer, String, DateTime
from datetime import datetime

Base = declarative_base()

class User(Base):
    __tablename__ = "users"

    id = Column(Integer, primary_key=True, autoincrement=True)
    name = Column(String(100), nullable=False)
    email = Column(String(100), unique=True, nullable=False, index=True)
    age = Column(Integer)
    created_at = Column(DateTime, default=datetime.utcnow)
    updated_at = Column(DateTime, default=datetime.utcnow, onupdate=datetime.utcnow)

# 创建表(生产环境请使用 Alembic 进行迁移管理)
Base.metadata.create_all(bind=pg_engine)

4.2 ORM-based CRUD

Also wrap the session with a context manager to keep the style consistent:

from contextlib import contextmanager

@contextmanager
def pg_orm_session():
    session = PgSession()
    try:
        yield session
        session.commit()
    except Exception as e:
        session.rollback()
        logging.error(f"ORM 操作失败: {e}")
        raise
    finally:
        session.close()

# 创建用户
def create_orm_user(name, email, age):
    with pg_orm_session() as session:
        user = User(name=name, email=email, age=age)
        session.add(user)
        session.flush()   # 在不提交事务的情况下获取 ID
        return user.id

# 条件查询与排序
def get_young_users(min_age=18, max_age=30, limit=10):
    with pg_orm_session() as session:
        return session.query(User)\
                     .filter(User.age.between(min_age, max_age))\
                     .order_by(User.created_at.desc())\
                     .limit(limit)\
                     .all()

Compared with native SQL, ORM code is closer to business logic and easier to maintain when fields change.


5. Avoid pitfalls and best practices

5.1 SQL injection protection (top priority)

**Never concatenate user input into a SQL string! ** This is a safety red line.

✅ Correct approach - use parameterized queries:

cursor.execute("SELECT * FROM users WHERE name = %s", (user_input_name,))

❌ Wrong practice - string concatenation:

cursor.execute(f"SELECT * FROM users WHERE name = '{user_input_name}'")

ORM will automatically escape parameters, so you can use them with confidence.

5.2 Key points of transaction management

Native driver and SQLAlchemy defaultautocommit=False, which is actually a good thing - only if you explicitly callcommit()Only then will the data actually be written. Any exception should be called promptlyrollback(), to ensure data consistency. The context manager provided in this article already has this logic built in and can be used directly.

5.3 Index design principles

  • Common query conditions are given priority to build indexes: such asWHERE, JOIN, ORDER BYfields in .
  • Automatic index creation for unique constraints: in the above modelemailThis is a typical example.
  • Avoid building indexes on small tables or fields that are updated extremely frequently, as the gain may outweigh the loss.
  • PostgreSQL supportCREATE INDEX CONCURRENTLY, indexes can be created online without locking the table.

1. **Lightweight or learning project**: Use **native driver + connection pool**, simple and direct, with few dependencies. 2. **Medium-sized or above or requiring cross-database**: Use **SQLAlchemy ORM** directly to reduce maintenance costs. 3. **Any production environment**: Be sure to **environment variables manage sensitive information** and **enable connection pooling**.

Summarize

The core logic of Python connecting MySQL and PostgreSQL is basically the same. The differences are mainly reflected in the driver syntax and the unique features of PostgreSQL (such asUPSERTJSONB). Remember the three key points of connection pooling, parameterized queries, and explicit transaction management, and you will be able to write efficient, safe, and maintainable database code. I hope this guide can serve as a quick reference for your daily development.