Skip to content

Feature Request: Support arbitrary io buffers in open_database #232

@seemanne

Description

@seemanne

In some environments (cloud) its sometimes not optimal to save/load files from disk and instead load them in remotely. In such cases you can get an io Buffer object (like io.BytesIO or io.BufferedReader) which behaves exactly like a file without being a file on disk.

The library in its current state theoretically supports using such objects when using MODE_FD but in practice it doesn't allow this because L86 in reader.py tries to access the name attribute of the buffer which does not exist because it doesn't wrap a file. Python also doesn't allow you to set that attribute on a buffer yourself by simply assigning it.

(

filename = database.name # type: ignore[union-attr]
)

Considering the filename defined in L86 is only used in error messages, is it possible to add a fix that defaults to some value if the attribute does not exist for the buffer?

POC for the issue:

import io
import maxminddb

f = open(<your-mmdb-name-here>, "rb")
buf = io.BytesIO(f.read())

reader = maxminddb.open_database(buf, MODE_FD) # this fails '_io.BytesIO' object has no attribute 'name'

# similarly you can wrap into a BufferedReader

f = open(<your-mmdb-name-here>, "rb")
buf = io.BytesIO(f.read())
stream = io.BufferedReader(buf)

reader = maxminddb.open_database(stream, MODE_FD) # this again fails for missing attribute

Note that just defaulting to something in L86 fixes this issue and allows the reader to work correctly.

I would be more than happy to contribute a fix for this if you want, it should be a very small change.

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions