mmap创建超过既定文件大小的memory-mapped file

问题

mmap针对Windows和Unix的版本在具体实现上有所不同，对于Windows版本，当length参数比file本身size大的时候，会自动扩展file为指定length大小；而Unix版本不支持自动扩展，即length只能小于等于size of file，如果超出size，则会报错。

解决方案

The unix version of mmap doesn’t grow the file automatically, but can just write zeros to the file by yourself:

size = 1073741824 # bytes in 1GiB self.file = open("/path/demo.csv", "w+b") self.file.write("\0" * size) self.file.flush() self.mm = mmap.mmap(self.file.fileno(), size, access= mmap.ACCESS_WRITE)

But this takes too long to finish, it can be improved by :

with open("file.to.create", "wb") as out: out.seek((1024 * 1024 * 1024) - 1) out.write('\0')

Reference

Why can’t I create a new file with mmap of known size?create file of particular size in python