|
|
---layout: posttitle: 如何制作一个“完整”的博客压缩包?tags: [压缩包, Quine, LZMA2, 7-Zip]---
让AI做出真正的“完整”将不再是难事……<!--more-->
# 起因
在上次[用AI制作了各种东西](/2026/09/01/vibe-coding2.html)之后,我已经完全理解了AI确实是无所不能的。既然如此,那就让它帮我解决曾经未能解决的事情吧? 去年,我为了让下载全站压缩包的按钮不断链,让这个压缩包也包含它本身而研究了[ZIP Quine](/2025/09/01/quine.html),但限于DEFLATE的回溯窗口大小没能做到……但那是人做的东西,人还是太弱小了,现在换AI来试试,也许一切将变得不一样?
# 制作基于LZMA2的博客压缩包
首先,我把[Ruben Van Mello](https://github.com/ruvmello)写的那篇论文《[A Generator for Recursive Zip Files](https://www.mdpi.com/2076-3417/14/21/9797)》以及生成器[zip-quine-generator](https://github.com/ruvmello/zip-quine-generator)发给了AI,问它这个限制是不是真的,是不是真的无法创造出超过32KiB的ZIP Quine?没过多久它分析完了,告诉我确实存在这样的问题,但并不是无法解决的,最简单的办法就是换个回溯窗口更大的压缩算法。我看了一下它给我列出的几个算法,看起来LZMA2和zstd比较符合要求。不过zstd感觉不是很知名,一般的解压软件应该处理不了,所以我就选择让它基于LZMA2来制作了。 不过LZMA2只是算法,还得选个容器,考虑到7z算是最出名,而且LZMA2本来就是7-Zip的作者开发的,所以就直接让它做7z版本的了。另外因为[Nate Choe](https://github.com/NateChoe1)做了通过扩展欧几里得求逆元来计算CRC32的[PR](https://github.com/ruvmello/zip-quine-generator/pull/3),所以我也告诉它要用这种方法在多项式时间内解出CRC32的值,而不是爆破。就这样AI花了两个小时左右成功把代码写出来了,效果非常完美,另外它还偷懒地将计算CRC32的算法改成了高斯消元法😆,因为7z格式有3个相互影响的CRC32值,用扩展欧几里得会更复杂一点,不过对7z来说不写入文件的CRC32也不影响,只是会不校验罢了,至少它还是按我要求做了……但其实这个问题也并不是完全不能用扩展欧几里得算法,后来我又强烈要求了一下,AI还是给我写出来了。以下是代码片段,有兴趣的人可以参考一下(不过这个看起来写的就是很复杂,所以我自己只用了高斯消元法): ```pythonRU_POLY = 0x104C11DB7 # x^32+x^26+x^23+...+1(含 x^32 首项)
def _find_probe(p): ret = 0 for i in range(64): if (1 << i) & p: ret = i return ret
def _mul_raw(p1, p2): """多项式乘法(无模)。""" ret = 0 probe = _find_probe(p1) for i in range(64): if p2 & (1 << i): assert probe + i < 64, "多项式乘法溢出" ret ^= p1 << i return ret
def _poly_divmod(dividend, divisor): """多项式长除法,返回 (商, 余)。""" probe = _find_probe(divisor) probe_bit = 1 << probe quot = 0 rem = dividend for i in range(63 - probe, -1, -1): if rem & (probe_bit << i): quot |= 1 << i rem ^= divisor << i return quot, rem
def _mul(p1, p2, mod): """p1*p2 % mod。""" return _poly_divmod(_mul_raw(p1, p2), mod)[1]
def _xgcd(p1, p2): """扩展欧几里得:返回 (k1,k2,gcd),满足 p1*k1+p2*k2=gcd。""" if _find_probe(p1) < _find_probe(p2): k1, k2, g = _xgcd(p2, p1) return k2, k1, g if p2 == 0: return p1, 0, p1 q, r = _poly_divmod(p1, p2) c1, c2, g = _xgcd(p2, r) return c2, c1 ^ _mul_raw(c2, q), g
def _minv(p, mod): """p 在环上的乘法逆元;不存在时返回 0。""" k1, _, g = _xgcd(p, mod) if g != 1: return 0 return _poly_divmod(k1, mod)[1]
def _bitrev32(x): """32 位比特反转。CRC 的输出循环冗余(binascii 反射)↔ 本环(非反射)互为 bit-reverse。""" return int(format(x & 0xFFFFFFFF, "032b")[::-1], 2)
def _ring_pow_x(k): """环内 x^k mod P(P=RU_POLY),二进制幂 O(log k)。变量列就是单个环幂。""" res, base, mod, e = 1, 2, RU_POLY, k while e: if e & 1: res = _mul(res, base, mod) base = _mul(base, base, mod) e >>= 1 return res
def solve_crc_system(files): """解析解 CRC 定点系统
- 常数列 :对“变量清零”的数据做一次 binascii.crc32,再 bit-reverse 32 位 (binascii 反射 CRC 与本环非反射 CRC 互为 bit-reverse,见 _bitrev32)。 - 变量列 :单个环幂 x^(32 + 8*(len-pos-4)),二进制幂 O(log N) 一步算出; 同一变量出现在多处时按位异或累加。 各自 O(#rows) 次 CRC + 少量环幂即可建出整矩阵,Gauss 消元不变。 """ n = len(files) mod = RU_POLY matrix = [[0] * (n + 1) for _ in range(n)]
for file in range(n): data = files[file][0] offsets = files[file][1] N = len(data) zero = bytearray(data) for pos in offsets: # 未知量占 4 字节,清零后求常数项 zero[pos:pos + 4] = b"\x00\x00\x00\x00" matrix[file][n] = _bitrev32(crc32(bytes(zero))) for pos, vid in offsets.items(): rest = N - pos - 4 # 变量被 4 字节(x^32)+ 其后字节(x^8/个)推到底 matrix[file][vid] ^= _ring_pow_x(32 + 8 * rest) matrix[file][file] ^= 1
for sr in range(n): ex = sr while ex < n and matrix[ex][sr] == 0: ex += 1 if ex == n: raise RuntimeError("CRC 多项式系统奇异 @row%d" % sr) if ex != sr: matrix[sr], matrix[ex] = matrix[ex], matrix[sr] inv = _minv(matrix[sr][sr], mod) for er in range(sr + 1, n): mult = _mul(inv, matrix[er][sr], mod) for ec in range(n + 1): matrix[er][ec] ^= _mul(mult, matrix[sr][ec], mod)
res = [0] * n for row in range(n - 1, -1, -1): v = matrix[row][n] for col in range(row + 1, n): v ^= _mul(res[col], matrix[row][col], mod) res[row] = _mul(v, _minv(matrix[row][row], mod), mod)
return [_impl_val(res[i]) for i in range(n)]
def _impl_val(math_val): imp = 0 for bit in range(32): if math_val & (1 << bit): imp |= 1 << (31 - bit) return imp
class BlogQuine: def solve_crc(self, F, lay, seed): T, h, d = lay["T"], lay["h"], lay["d"] k, n_sub, crc_base = lay["k"], lay["n_sub"], lay["crc_base"] hoff = lay["total"] - h vout_f = lay["total"] - 2 * T hcopy = vout_f + (T - h) # 已知 CRC:seed + 各文件(quine 的是未知量 D) known = [crc32(seed)] + [crc32(data) for _, data in self.files] assert len(known) == n_sub - 1 for i, e in enumerate(known): struct.pack_into("<I", F, hoff + crc_base + 4 * i, e) struct.pack_into("<I", F, hcopy + crc_base + 4 * i, e) # 未知量 D N S:D=整个文件、N=头部 F[hoff:hoff+h]、S=签名 F[12:32] dpos = crc_base + 4 * (n_sub - 1) ncopy = 32 + 3 * k + d # C1k 载荷里 F[0:35) 副本起点 groups = [[hoff + dpos, hcopy + dpos], # D 出现两处 [28, ncopy + 28], # N [8, ncopy + 8]] # S
# 构造 3 个“文件”喂给解析解系统: # 文件0 = 整个 F,三个未知量各自出现在 groups 两处 # 文件1 = 头部块,只有 D 出现在相对 dpos(另一处 hcopy+dpos 在头外) # 文件2 = 签名块,只有 N 出现在相对 16(28-12) whole = bytes(F) hdr = bytes(F[hoff:hoff + h]) sig = bytes(F[12:32]) f0_off = {hoff + dpos: 0, hcopy + dpos: 0, 28: 1, ncopy + 28: 1, 8: 2, ncopy + 8: 2} f1_off = {dpos: 0} f2_off = {28 - 12: 1} vals = solve_crc_system([(whole, f0_off), (hdr, f1_off), (sig, f2_off)])
for g in range(3): for p in groups[g]: F[p:p + 4] = struct.pack("<I", vals[g]) real = (crc32(F), crc32(F[hoff:hoff + h]), crc32(F[12:32])) assert real == tuple(vals), "CRC 定点失败: %s != %s" % (real, vals) return vals```
于是我第一时间就把原来的TGZ压缩命令换掉,换成了AI给我写的[blogquine.py](https://github.com/Mabbs/mabbs.github.io/blob/master/_tools/blogquine.py),现在就可以通过[这里](https://mayx.eu.org/MayxBlog.7z)下载到“完整”包含我博客所有内容的压缩包了。 不过唯一的问题就是这样做出来的压缩包并没有压缩😂,相当于给做成了普通的归档了。当然我的博客本身倒是不大,没压缩也多不了多少空间,但相比于能做出“完整”的效果来说,这点浪费的空间也是小问题了。 ## 基于博客压缩包的离线模式(2026.10.03更新)
既然现在的AI什么都能做了,那上次我因为技术不足仅仅只是用[Service Worker做了个代理](/2025/08/01/sw-proxy.html),现在也可以升级成下载全站之后让全站离线的功能吧?不过之前做的压缩包并没有压缩,导致整个压缩包有接近20MiB的大小,后来我仔细想了一下我的需求,我只是想让下载全站压缩包的按钮不断链,并没有说解压后的文件一定就必须是它本身,所以我只要用相同的文件名再压缩一次应该就可以了,这样既能满足我的想法,又可以让其他人下载的时候不会花费太多的时间。 在我直接压缩之后,整个压缩包占用大概9MiB的空间,其实我已经很满意了,不过压之前是18MiB,也就是说压缩率是50%……虽然也不是不能接受,但我在想这是不是合理的结果呢?我让AI分析了一下50%的压缩率正不正常,它看了一眼之后告诉我文本部分的10MiB压缩完之后有1MiB,图片因为压不动所以直接占用了8MiB😅,这时候我才意识到原来我一直存着一堆占着大量空间的垃圾,而且还是用的PNG/JPG这种压缩率很低的格式……考虑到兼容性和压缩效果,我把占用比较大的图片转换为了WebP格式,最终整个压缩包就只占用了4MiB的空间,一口气又节省了50%的空间。 最后做完对压缩包的处理之后,我直接让AI给我基于那个压缩包制作离线模式,它轻轻松松就[做出来](https://mayx.eu.org/offline.html)了,最终使用的是[libarchive库](https://github.com/nika-begiashvili/libarchivejs),在启用离线模式之后整个网站的速度简直就是0延迟,点击就能瞬间跳转,效果算是相当不错了。 ## 受大佬启发制作了能压缩的7z Quine(2026.10.05更新)
做完博客压缩包之后我突发奇想,虽然东西是AI做的,但也是在我的手中做出了世界上第一个LZMA2 Quine,那么如果我向全世界第一个制作ZIP Quine的人展示一下我的作品会怎么样呢?目前可以知道,世界上第一个制作ZIP Quine的人是[Erling Ellingsen](https://infosec.exchange/@steike),所以我向他发了个帖子[展示了一下](https://infosec.exchange/@mayx/117385801954308033)。大佬不愧是大佬,一眼就能看出理论上能压缩的7z Quine是可以做出来的,对我来说,我是完全不知道7-Zip居然还支持链式编解码,能可以通过“double lzma2”的方式做到既可以自包含、又能真正被压缩的压缩包。既然大佬说理论上可以,那我自然是相信的,于是就让AI给我使劲折腾想办法把它做出来。在折腾了一晚上之后AI终于做出来了,真是强得可怕: ```python# ============================================================
# 双层 LZMA2 的 7z quine("double lzma2")
## F = S32 ‖ P ‖ H,folder 的 coder 链 = [LZMA2(外层), LZMA2(内层)],
# 解码管线:P --外层--> I --内层--> U。
## I = Cseg ‖ LBseg ‖ Mseg ‖ 0x00
# Cseg : 真实 LZMA2 对 content 的压缩流(内容真正的压缩数据 C)
# LBseg: 内层 store 链,载荷 = seed(seed 内含一份 C 的参照副本)
# Mseg : 内层 match 链,输出 quine 区(= F 的逐字节副本)
# U = content ‖ seed ‖ F
## 外层把 I 再压一遍:Cseg 用 store 链原样承载;seed 里的 C 参照副本与 Cseg
# 逐字节相同,被外层的 rep0 回溯匹配消掉,于是成品里只剩一份压缩数据。
# quine 区回引 P 的字节时,源全在 seed 里(C 副本 / 头样本表 / 机械串)。
# ============================================================
def lzma_chunk(tokens, out_pos): """编一个 LZMA chunk(0xC0: state+props reset,无 dict reset,无 end marker)。 match token 的编码字节只取决于 (dist,len,pos),与历史内容无关(不维护输出历史)。 解压长度用 21 位(控制字节低 5 位存高 5 位),故单 chunk 输出上限 2 MiB。""" enc = LzmaEncoder() enc.pos = out_pos total = 0 for t in tokens: if t[0] == "m": enc.match(t[1], t[2]) total += t[2] else: enc.rep_match(t[1]) total += t[1] data = enc.finish() u = total - 1 c = len(data) - 1 assert 0 <= u < (1 << 21) and 0 <= c < 65536, (total, len(data)) hdr = bytes([0xC0 | ((u >> 16) & 0x1F)]) + (u & 0xFFFF).to_bytes(2, "big") + \ (c & 0xFFFF).to_bytes(2, "big") + bytes([PROPS_BYTE]) return hdr + data, total
LZ2_MAXU = 1 << 21 # LZMA2 单个 LZMA chunk 的解压上限
def lzma2_raw(data, dict_size, preset=6): """FORMAT_RAW + LZMA2 压缩;去掉结尾 0x00 便于后面继续追加 chunk。""" c = lzma.LZMACompressor( format=lzma.FORMAT_RAW, filters=[{"id": lzma.FILTER_LZMA2, "dict_size": dict_size, "preset": preset}]) out = c.compress(data) + c.flush() assert out and out[-1] == 0x00, "LZMA2 流应以 0x00 结尾" return out[:-1]
def split_len(n, cap): """按 cap 上限切分 n 字节。末片若为 1 字节则从前一片借 1(match 最短为 2)。""" Ls, rem = [], n while rem > cap: Ls.append(cap) rem -= cap if rem == 1 and Ls: Ls[-1] -= 1 rem = 2 Ls.append(rem) return Ls
class DoubleQuine(BlogQuine): """双层 LZMA2 版本:folder = [LZMA2(外层), LZMA2(内层)]。"""
def entries(self): """U 的顺序是 内容文件… ‖ seed ‖ quine,条目顺序必须与之对应。""" ent = [{"name": r, "dir": isd} for r, isd in self.walk_entries] ent.append({"name": self.seed_name, "dir": False}) ent.append({"name": self.quine_name, "dir": False}) return ent
# ---------- 头部:双 coder + BindPair ---------- def build_header(self, n, total, d, entries, sub_sizes, size_I): n_sub = len(sub_sizes) N = len(entries) h = bytearray() h += b"\x01\x04" # kHeader, kMainStreamsInfo h += b"\x06" + varint(0) + varint(1) + b"\x09" + varint(n) + b"\x00" h += b"\x07" # kUnpackInfo h += b"\x0b" + varint(1) + b"\x00" # kFolder: 1, local h += varint(2) # NumCoders = 2 h += b"\x21\x21" + varint(1) + bytes([self.dict_prop_out]) # coder0 = 外层 h += b"\x21\x21" + varint(1) + bytes([self.dict_prop_in]) # coder1 = 内层 h += varint(1) + varint(0) # BindPair = (InIndex=1, OutIndex=0) h += b"\x0c" + varint(size_I) + varint(d + total) # 两个 coder 的输出大小 h += b"\x00" h += b"\x08" # SubStreamsInfo h += b"\x0d" + varint(n_sub) h += b"\x09" + b"".join(varint(s) for s in sub_sizes[:-1]) h += b"\x0a" + b"\x01" crc_base = len(h) h += b"\x00" * (4 * n_sub) h += b"\x00\x00" h += b"\x05" + varint(N) # FilesInfo bits = bytearray((N + 7) // 8) for i, e in enumerate(entries): if e["dir"]: bits[i // 8] |= 1 << (7 - i % 8) h += b"\x0e" + varint(len(bits)) + bytes(bits) mt = b"\x01\x00" + struct.pack("<Q", self.mtime_ft) * N h += b"\x14" + varint(len(mt)) + mt names = bytearray(b"\x00") for e in entries: names += e["name"].encode("utf-16-le") + b"\x00\x00" h += b"\x11" + varint(len(names)) + bytes(names) attrs = b"".join(struct.pack("<I", 0x10 if e["dir"] else 0x20) for e in entries) h += b"\x15" + varint(2 + len(attrs)) + b"\x01\x00" + attrs h += b"\x00\x00" return bytes(h), crc_base
# ---------- 布局:三层坐标的不动点迭代 ---------- def layout(self): content = self.content dc = len(content) dict_c = max(1 << 20, 1 << max(20, dc.bit_length())) C = lzma2_raw(content, dict_c) self._C = C # 内容压缩只做一次,finalize 复用 lc = len(C) # 迭代变量:Y = plant 长度, H = 头部长度, M = Mseg 字节数 Y, H, M = 96, 256, 256 seen = set() for _ in range(200): ds = lc + 32 + H + Y # seed = C副本‖S32‖H‖plant d = dc + ds Ls_LB = split_len(ds, CHUNK) lLB = 3 * len(Ls_LB) + ds off_in = lc + lLB M1 = M + 1 # Mseg ‖ 内层流终止符 0x00 Ls_M = split_len(M1, CHUNK) Li = off_in + M1 self.dict_prop_out, _ = dict_prop_for(max(Li, 1 << 20)) self.dict_prop_in, _ = dict_prop_for(d + 32 + H + 8192) st = self._build_streams(C, lc, ds, d, Ls_LB, lLB, off_in, Ls_M, M1, H) if st is None: Y += 8 continue mseg = self._build_mseg(st, d, lc, ds, H, M) if mseg is None: Y += 8 continue M_new = len(mseg) Y_new = len(st["hdr_bytes"]) + len(st["mech"]) + M_new + 2 total = st["n"] + 32 + H file_sizes = [len(x) for _, x in self.files] sub_sizes = file_sizes + [ds] + [total] entries = self.entries() hdr, crc_base = self.build_header(st["n"], total, d, entries, sub_sizes, Li) H_new = len(hdr) key = (Y_new, H_new, M_new) if key in seen: # 再次见到同一组值 = 已收敛 Y, H, M = Y_new, H_new, M_new break seen.add(key) Y, H, M = Y_new, H_new, M_new else: raise RuntimeError("double layout 不收敛") return {"Y": Y, "H": H, "M": M}
# ---------- 定稿:用收敛后的 (Y, H, M) 重算全部结构 ---------- def finalize(self, Y, H, M): content = self.content dc = len(content) C = self._C # layout 里已压缩过 lc = len(C) ds = lc + 32 + H + Y d = dc + ds Ls_LB = split_len(ds, CHUNK) lLB = 3 * len(Ls_LB) + ds off_in = lc + lLB M1 = M + 1 Ls_M = split_len(M1, CHUNK) Li = off_in + M1 self.dict_prop_out, ds_out = dict_prop_for(max(Li, 1 << 20)) self.dict_prop_in, ds_in = dict_prop_for(d + 32 + H + 8192) st = self._build_streams(C, lc, ds, d, Ls_LB, lLB, off_in, Ls_M, M1, H) mseg = self._build_mseg(st, d, lc, ds, H, M) assert len(mseg) == M, "Mseg %d != %d" % (len(mseg), M) total = st["n"] + 32 + H file_sizes = [len(x) for _, x in self.files] sub_sizes = file_sizes + [ds] + [total] entries = self.entries() hdr, crc_base = self.build_header(st["n"], total, d, entries, sub_sizes, Li) assert len(hdr) == H, "header %d != %d" % (len(hdr), H) plant = st["hdr_bytes"] + bytes(st["mech"]) + mseg + b"\x00\x00" assert len(plant) == Y, "plant %d != %d" % (len(plant), Y) return {"C": C, "lc": lc, "dc": dc, "ds": ds, "d": d, "Li": Li, "ochunks": st["ochunks"], "hdr_tab": st["hdr_tab"], "mseg": mseg, "M": M, "H": H, "n": st["n"], "total": total, "header": hdr, "crc_base": crc_base, "plant": plant, "dict_size_in": ds_in, "dict_size_out": ds_out, "n_sub": len(sub_sizes)}
# ---------- 外层流 P 的构造 ---------- def _build_streams(self, C, lc, ds, d, Ls_LB, lLB, off_in, Ls_M, M1, H): hdr_tab, hdr_idx = [], {}
def hidx(b): assert len(b) == 3 and b[0] in (0x01, 0x02), b.hex() if b not in hdr_idx: hdr_idx[b] = len(hdr_tab) hdr_tab.append(b) return hdr_idx[b]
ochunks = [] f, io = 32, 0 mech = bytearray()
def put_lzma(tokens, ioff): blk, t = lzma_chunk(tokens, ioff) ochunks.append({"kind": "lzma", "foff": f, "size": len(blk), "ioff": ioff, "olen": t, "bytes": blk, "mech": len(mech)}) mech.extend(blk) return len(blk), t
def put_store(ioff, olen, hi, src): ochunks.append({"kind": "store", "foff": f, "size": 3 + olen, "ioff": ioff, "olen": olen, "hdr": hi, "src": src}) return 3 + olen
# 段 1:store 链承载 Cseg(I[0:lc] = C 的字节;源 = seed 的 C 参照副本) Ls_C = split_len(lc, CHUNK) c_off = 0 for L in Ls_C: f += put_store(io, L, hidx(store_hdr(L, first=(c_off == 0))), ("seed", c_off)) io += L c_off += L assert io == lc
# 段 2:LBseg(I[lc : lc+lLB])—— 内层 store 链,载荷 = seed lb_off = lc seed_off = 0 for j, L in enumerate(Ls_LB): lb_hdr = store_hdr(L, first=False) if j == 0 or L != CHUNK: # 没有可回引的相同样本,用外层 store chunk 原样携带这 3 字节 f += put_store(lb_off, 3, hidx(store_hdr(3, first=False)), ("tab", hidx(lb_hdr))) io += 3 else: # 与 chunk0 的头逐字节相同,直接从 I[lc:lc+3] 复制 n, t = put_lzma([("m", lb_off - lc, 3)], lb_off) assert t == 3 f += n io += 3 # 载荷:C 参照副本区走 rep0 回溯匹配(去重发生处),其余走 store pay = lb_off + 3 i = 0 while i < L: if seed_off + i < lc: a = min(L - i, lc - (seed_off + i), LZ2_MAXU) dist = (pay + i) - (seed_off + i) n, t = put_lzma(matches_for(dist, a), pay + i) assert t == a f += n io += a i += a else: a = min(L - i, LZ2_MAXU) f += put_store(pay + i, a, hidx(store_hdr(a, first=False)), ("seed", seed_off + i)) io += a i += a lb_off += 3 + L seed_off += L assert lb_off == lc + lLB and io == lc + lLB
# 段 3:store 链承载 Mseg ‖ 0x00(0x00 是内层流的终止符) m_off = 0 for L in Ls_M: f += put_store(off_in + m_off, L, hidx(store_hdr(L, first=False)), ("mseg", m_off)) io += L m_off += L # 外层流自己的终止符 0x00 ochunks.append({"kind": "term", "foff": f, "size": 1, "ioff": io, "olen": 0, "bytes": b"\x00"}) f += 1 return {"ochunks": ochunks, "mech": mech, "hdr_tab": hdr_tab, "hdr_bytes": b"".join(hdr_tab), "n": f - 32}
# ---------- 内层 Mseg:把 quine 区的每个 F 区段映射成 match 链 ---------- def _build_mseg(self, st, d, lc, ds, H, M): dc = d - ds s_S32 = dc + lc # seed 里 S32 副本 s_H = dc + lc + 32 # seed 里 H 副本 s_tab = dc + lc + 32 + H # seed 里头样本表 s_mech = s_tab + len(st["hdr_bytes"]) # seed 里机械串 s_mseg = s_mech + len(st["mech"]) # seed 里 Mseg 副本 total = st["n"] + 32 + H hoff = 32 + st["n"] segs = [(0, 32, s_S32)] # F[0:32) 签名头 for c in st["ochunks"]: if c["kind"] == "store": segs.append((c["foff"], 3, s_tab + 3 * c["hdr"])) src = c["src"] if src[0] == "seed": u = dc + src[1] elif src[0] == "tab": u = s_tab + 3 * src[1] else: # ("mseg", off) u = s_mseg + src[1] segs.append((c["foff"] + 3, c["olen"], u)) elif c["kind"] == "lzma": segs.append((c["foff"], c["size"], s_mech + c["mech"])) else: # term:并入前一个载荷段 pf, pl, pu = segs[-1] segs[-1] = (pf, pl + 1, pu) segs.append((hoff, H, s_H)) # F[hoff:] 头部 pos = 0 for (fo, ln, u) in segs: assert fo == pos, "段不连续: %d != %d" % (fo, pos) assert ln >= 2, "段太短 %d @%d" % (ln, fo) pos += ln assert pos == total, "段总长 %d != total %d" % (pos, total) # 生成 match 链,按 2 MiB 上限切成多个 0xC0 chunk mseg = bytearray() out_pos = d cur, cur_len = [], 0 chunks = [] for (fo, ln, u) in segs: dist = (d + fo) - u assert 0 < dist, "dist=%d @%d" % (dist, fo) toks = matches_for(dist, ln) if cur_len + ln > LZ2_MAXU - MAX_MATCH: chunks.append((cur, out_pos, cur_len)) out_pos += cur_len cur, cur_len = [], 0 cur.extend(toks) cur_len += ln if cur: chunks.append((cur, out_pos, cur_len)) out_pos += cur_len assert out_pos == d + total, "Mseg 输出 %d != %d" % (out_pos, d + total) for (toks, op, cl) in chunks: blk, t = lzma_chunk(toks, op) assert t == cl mseg.extend(blk) return bytes(mseg)
# ---------- F 中承载 seed[soff] 的那个字节的偏移 ---------- def _f_off_of_seed(self, lay, soff): """返回 (F 偏移, 该 store 载荷内从 soff 起还剩多少字节)。 seed_head 超过 64 KiB 时会被切成多片,所以调用方要循环推进。""" for c in lay["ochunks"]: if c["kind"] == "store" and c["src"][0] == "seed": s0 = c["src"][1] if s0 <= soff < s0 + c["olen"]: return c["foff"] + 3 + (soff - s0), s0 + c["olen"] - soff raise RuntimeError("seed 偏移 %d 未被 store 载荷覆盖" % soff)
def _f_pos_list(self, lay, soff, length): """seed[soff:soff+length] 在 F 中对应的字节位置列表(可能跨多片)。""" res = [] while length > 0: p, avail = self._f_off_of_seed(lay, soff) a = min(length, avail) res.extend(range(p, p + a)) soff += a length -= a return res
# ---------- 装配 ---------- def assemble(self, lay): total, H = lay["total"], lay["H"] hoff = total - H C, plant = lay["C"], lay["plant"] F = bytearray(total) # pass A: 签名头 + 所有非 store 载荷字节 F[0:6] = SIG F[6:8] = VER struct.pack_into("<Q", F, 12, lay["n"]) struct.pack_into("<Q", F, 20, H) for c in lay["ochunks"]: if c["kind"] == "store": F[c["foff"]:c["foff"] + 3] = lay["hdr_tab"][c["hdr"]] else: F[c["foff"]:c["foff"] + c["size"]] = c["bytes"] F[hoff:hoff + H] = lay["header"] # pass B: seed = C副本 ‖ S32副本 ‖ H副本 ‖ plant seed = bytearray() seed += C seed += bytes(F[0:32]) seed += bytes(F[hoff:hoff + H]) seed += plant assert len(seed) == lay["ds"], "seed %d != %d" % (len(seed), lay["ds"]) # pass C: 各 store 的载荷 msegz = lay["mseg"] + b"\x00" for c in lay["ochunks"]: if c["kind"] != "store": continue p = c["foff"] + 3 src, L = c["src"], c["olen"] if src[0] == "seed": F[p:p + L] = seed[src[1]:src[1] + L] elif src[0] == "tab": F[p:p + 3] = lay["hdr_tab"][src[1]] else: F[p:p + L] = msegz[src[1]:src[1] + L] return F, bytes(seed)
# ---------- CRC 定点(128x128 GF(2):seed/quine/Next/Start 四个 CRC) ---------- def solve_crc(self, F, lay, C, plant): H, hoff = lay["H"], lay["total"] - lay["H"] n_sub, crc_base = lay["n_sub"], lay["crc_base"] lc = lay["lc"] # seed 里 S32 / H 副本在 F 中的镜像(可能跨多片,逐字节列出) scopy0 = self._f_pos_list(lay, lc, 32) Fv = memoryview(F)
def seed_crc(): v = crc32(C) v = crc32(Fv[0:32], v) # S32 副本 v = crc32(Fv[hoff:hoff + H], v) # H 副本 return crc32(plant, v)
def targets(): return (seed_crc(), crc32(Fv), crc32(Fv[hoff:hoff + H]), crc32(Fv[12:32]))
# 已知:各内容文件的 CRC,写本体 + seed 里 H 副本的镜像两处 known = [crc32(data) for _, data in self.files] for i, e in enumerate(known): slot = crc_base + 4 * i b = struct.pack("<I", e) for k in range(4): F[hoff + slot + k] = b[k] mir = self._f_pos_list(lay, lc + 32 + slot, 4) assert len(mir) == 4 for k in range(4): F[mir[k]] = b[k] # 未知量:D_seed(seed 的 CRC)、D(quine 的 CRC)、N、S # 每个未知量 4 字节,每字节有"本体 + seed 镜像"两个落点 qs = crc_base + 4 * len(known) # seed 子流的 CRC 槽 q = crc_base + 4 * (n_sub - 1) # quine 子流的 CRC 槽 groups = [] for slot in (qs, q): groups.append([list(range(hoff + slot, hoff + slot + 4)), self._f_pos_list(lay, lc + 32 + slot, 4)]) groups.append([list(range(28, 32)), scopy0[28:32]]) # NextHeaderCRC groups.append([list(range(8, 12)), scopy0[8:12]]) # StartHeaderCRC nb = 32 * len(groups) base = targets() basevec = 0 for i, v in enumerate(base): basevec |= v << (32 * i) cols = [] for g in range(len(groups)): own, mir = groups[g] for kb in range(32): bi, bb = kb // 8, kb % 8 poss = [own[bi], mir[bi]] for p in poss: F[p] ^= (1 << bb) t = targets() for p in poss: F[p] ^= (1 << bb) delta = 0 for i, v in enumerate(t): delta |= (v ^ base[i]) << (32 * i) cols.append(delta) eqs = [] for i in range(nb): coeff = 0 for j in range(nb): if (cols[j] >> i) & 1: coeff |= 1 << j coeff ^= 1 << i eqs.append([coeff, (basevec >> i) & 1]) for j in range(nb): p = next((i for i in range(j, nb) if (eqs[i][0] >> j) & 1), None) if p is None: raise RuntimeError("GF(2) 奇异 @bit%d" % j) eqs[j], eqs[p] = eqs[p], eqs[j] for i in range(nb): if i != j and ((eqs[i][0] >> j) & 1): eqs[i][0] ^= eqs[j][0] eqs[i][1] ^= eqs[j][1] x = 0 for i in range(nb): assert eqs[i][0] == (1 << i) if eqs[i][1]: x |= 1 << i vals = [(x >> (32 * i)) & 0xFFFFFFFF for i in range(len(groups))] for g in range(len(groups)): own, mir = groups[g] b = struct.pack("<I", vals[g]) for k in range(4): F[own[k]] = b[k] F[mir[k]] = b[k] t = targets() assert t == tuple(vals), "CRC 定点失败: %s != %s" % (t, vals) return vals
def build(self): l0 = self.layout() lay = self.finalize(l0["Y"], l0["H"], l0["M"]) F, seed = self.assemble(lay) crcs = self.solve_crc(F, lay, lay["C"], lay["plant"]) # CRC 定点会改写 H / S32,而 seed 是它们的镜像,定点后需重建 hoff = lay["total"] - lay["H"] seed = lay["C"] + bytes(F[0:32]) + bytes(F[hoff:hoff + lay["H"]]) \ + lay["plant"] return bytes(F), lay, seed, crcs
def verify_double(F, lay, files, seed, content): """串联两个原始 LZMA2 解码器还原 U,逐段校验。""" d, total = lay["d"], lay["total"] dec_o = lzma.LZMADecompressor( format=lzma.FORMAT_RAW, filters=[{"id": lzma.FILTER_LZMA2, "dict_size": lay["dict_size_out"]}]) I = dec_o.decompress(F[32:32 + lay["n"]]) ok = True if len(I) != lay["Li"]: print("[verify] 外层输出 %d != I %d" % (len(I), lay["Li"])) return False dec_i = lzma.LZMADecompressor( format=lzma.FORMAT_RAW, filters=[{"id": lzma.FILTER_LZMA2, "dict_size": lay["dict_size_in"]}]) U = dec_i.decompress(I) exp = d + total if len(U) != exp: print("[verify] 内层输出 %d != %d" % (len(U), exp)) return False dc, ds = lay["dc"], lay["ds"] if U[:dc] != content: print("[verify] 内容不匹配") ok = False pos = 0 for rel, data in files: # 内容文件排在 U 的最前面 if U[pos:pos + len(data)] != data: print("[verify] 文件不匹配:", rel) ok = False pos += len(data) assert pos == dc if U[dc:dc + ds] != seed: print("[verify] seed 不匹配") ok = False if U[d:] != F: print("[verify] quine 自复制不匹配") ok = False print("[verify] 自解码: |I|=%d, |U|=%d, seed/文件/quine 全部匹配: %s" % (len(I), len(U), ok)) return ok``` 仅仅是大佬的一句话,AI就能做出来,实在是太可怕了。最终我试了一下,官方的7-Zip可以正常解压,但是Windows资源管理器不行……Windows资源管理器使用的也是[libarchive](https://github.com/libarchive/libarchive),似乎是因为libarchive只支持2个解码器,并且[第二个解码器只能是像BCJ这样的过滤器](https://github.com/libarchive/libarchive/blob/8bb3bbdc7b117a1e22086a2260f2087aafa90687/libarchive/archive_read_support_format_7zip.c#L1475)……明明7-Zip官方是可以解压的,不知道如果把这个问题当作BUG报告给libarchive他们会不会解决?但目前存在兼容性问题的话我博客压缩包就先不用这个方案了吧🥲…… ## 对TXZ格式的尝试
在做完7z格式的压缩包之后,我发现了一个问题,虽然7z确实很流行,但是在Linux下解压起来有点麻烦,7-Zip历史上主要面向Windows,Linux上长期以来更多依赖p7zip等第三方移植,因此生态集成度不如tar.xz,想要解压7z文件还得额外安装。 不过Linux下也有个支持LZMA2算法的压缩软件,那就是前些年出过[后门](https://tukaani.org/xz-backdoor/)的XZ Utils,配合tar就可以做出TXZ(tar.xz)文件,甚至用我[博客终端](https://mabbs.github.io/linux/)中的BusyBox也能解压。我想了一下反正有AI,干脆一句话让AI帮我把blogquine.py改成tar.xz格式的好了,结果倒也没费多少功夫,AI就这样写出来了:
<details markdown="1"><summary markdown="span">Show Code</summary>
```python#!/usr/bin/env python3
# -*- coding: utf-8 -*-
import argparseimport binasciiimport ioimport lzmaimport osimport structimport sysimport tarfileimport timeimport ctypesimport ctypes.util
# ============================================================
# Part 1: 最小 LZMA1 range coder(只做编码)
# ============================================================
kNumBitModelTotalBits = 11kBitModelTotal = 1 << kNumBitModelTotalBits # 2048kNumMoveBits = 5kTopValue = 1 << 24 # 0x1000000
kNumStates = 12kNumLitStates = 7kNumPosBitsMax = 4kNumLenToPosStates = 4kNumAlignBits = 4kEndPosModelIndex = 14kNumFullDistances = 1 << (kEndPosModelIndex >> 1) # 128kMatchMinLen = 2kNumLowLenBits = 3kNumMidLenBits = 3kNumHighLenBits = 8kNumLowLenSymbols = 1 << kNumLowLenBits # 8kNumMidLenSymbols = 1 << kNumMidLenBits # 8kNumPosSlotBits = 6
PROB_INIT = kBitModelTotal >> 1 # 1024
class RangeEncoder: """LZMA 的区间编码器。low 是 64 位(要容纳进位),range 是 32 位。"""
def __init__(self): self.low = 0 self.range = 0xFFFFFFFF self.cache = 0 self.cache_size = 1 # 初值 1 -> 第一个输出字节恒为 0x00 self.buf = bytearray()
def _shift_low(self): if (self.low & 0xFFFFFFFF) < 0xFF000000 or (self.low >> 32) != 0: temp = self.cache while True: self.buf.append((temp + (self.low >> 32)) & 0xFF) temp = 0xFF self.cache_size -= 1 if self.cache_size == 0: break self.cache = (self.low >> 24) & 0xFF self.cache_size += 1 self.low = ((self.low & 0xFFFFFFFF) << 8) & 0xFFFFFFFF
def encode_bit(self, probs, idx, bit): p = probs[idx] bound = (self.range >> kNumBitModelTotalBits) * p if bit == 0: self.range = bound probs[idx] = p + ((kBitModelTotal - p) >> kNumMoveBits) else: self.low += bound self.range -= bound probs[idx] = p - (p >> kNumMoveBits) while self.range < kTopValue: self.range = (self.range << 8) & 0xFFFFFFFF self._shift_low()
def encode_direct_bits(self, value, num_bits): for i in range(num_bits - 1, -1, -1): self.range >>= 1 if (value >> i) & 1: self.low += self.range while self.range < kTopValue: self.range = (self.range << 8) & 0xFFFFFFFF self._shift_low()
def bittree_encode(self, probs, off, num_bits, symbol): m = 1 for i in range(num_bits - 1, -1, -1): bit = (symbol >> i) & 1 self.encode_bit(probs, off + m, bit) m = (m << 1) | bit
def bittree_reverse_encode(self, probs, off, num_bits, symbol): m = 1 for i in range(num_bits): bit = symbol & 1 symbol >>= 1 self.encode_bit(probs, off + m, bit) m = (m << 1) | bit
def finish(self): for _ in range(5): self._shift_low() return bytes(self.buf)
def _pos_slot_and_bits(d): if d < 4: return d, 0, 0 for slot in range(4, 64): n = (slot >> 1) - 1 base = (2 | (slot & 1)) << n if d < base + (1 << n): return slot, n, d - base raise ValueError("distance too large: %d" % d)
# pb 必须与写进每个 LZMA2 chunk 头的 PROPS_BYTE(0x5D) 一致。
# lc/lp 只影响 literal 编码的概率表索引,而本编码器不实现 literal 路径,故无需常量。
PB = 2
class LzmaEncoder: """只实现 quine 用到的两条路径:match 与 rep0-match。
literal / rep-g1 / rep-g2 三条分支从未被构造,因此它们对应的概率表 (p_lit、p_is_rep_g1、p_is_rep_g2)以及 prev_byte 都不必存在。 reps 同理:编码器从不回读(rep0 的距离由解码器自行维护),故不保存。 """
def __init__(self): self.pos_mask = (1 << PB) - 1 self.rc = RangeEncoder() self.pos = 0 self.state = 0 n = PROB_INIT self.p_is_match = [n] * (kNumStates << kNumPosBitsMax) self.p_is_rep = [n] * kNumStates self.p_is_rep_g0 = [n] * kNumStates self.p_rep0_long = [n] * (kNumStates << kNumPosBitsMax) self.p_pos_slot = [n] * (kNumLenToPosStates << kNumPosSlotBits) self.p_spec_pos = [n] * (kNumFullDistances - kEndPosModelIndex) self.p_align = [n] * (1 << kNumAlignBits) self.p_len_choice = [n] * 2 self.p_len_low = [n] * (16 * kNumLowLenSymbols) self.p_len_mid = [n] * (16 * kNumMidLenSymbols) self.p_len_high = [n] * (1 << kNumHighLenBits) self.p_rep_len_choice = [n] * 2 self.p_rep_len_low = [n] * (16 * kNumLowLenSymbols) self.p_rep_len_mid = [n] * (16 * kNumMidLenSymbols) self.p_rep_len_high = [n] * (1 << kNumHighLenBits)
@property def pos_state(self): return self.pos & self.pos_mask
def _encode_len(self, choice, low, mid, high, length): ps = self.pos_state l = length - kMatchMinLen if l < kNumLowLenSymbols: self.rc.encode_bit(choice, 0, 0) self.rc.bittree_encode(low, ps << kNumLowLenBits, kNumLowLenBits, l) else: self.rc.encode_bit(choice, 0, 1) l -= kNumLowLenSymbols if l < kNumMidLenSymbols: self.rc.encode_bit(choice, 1, 0) self.rc.bittree_encode(mid, ps << kNumMidLenBits, kNumMidLenBits, l) else: self.rc.encode_bit(choice, 1, 1) self.rc.bittree_encode(high, 0, kNumHighLenBits, l - kNumMidLenSymbols)
def _write_dist(self, dist, lts): d = dist - 1 slot, n, low_bits = _pos_slot_and_bits(d) self.rc.bittree_encode(self.p_pos_slot, lts << kNumPosSlotBits, kNumPosSlotBits, slot) if slot >= 4: if slot < kEndPosModelIndex: base = (2 | (slot & 1)) << n off = base - slot - 1 self.rc.bittree_reverse_encode(self.p_spec_pos, off, n, low_bits) else: self.rc.encode_direct_bits(low_bits >> kNumAlignBits, n - kNumAlignBits) self.rc.bittree_reverse_encode(self.p_align, 0, kNumAlignBits, low_bits & ((1 << kNumAlignBits) - 1))
def match(self, dist, length): assert kMatchMinLen <= length <= 273, length ps = self.pos_state self.rc.encode_bit(self.p_is_match, (self.state << kNumPosBitsMax) + ps, 1) self.rc.encode_bit(self.p_is_rep, self.state, 0) lts = min(length - kMatchMinLen, kNumLenToPosStates - 1) self._encode_len(self.p_len_choice, self.p_len_low, self.p_len_mid, self.p_len_high, length) self._write_dist(dist, lts) self.state = 7 if self.state < kNumLitStates else 10 self.pos += length
def rep_match(self, length): assert kMatchMinLen <= length <= 273, length ps = self.pos_state self.rc.encode_bit(self.p_is_match, (self.state << kNumPosBitsMax) + ps, 1) self.rc.encode_bit(self.p_is_rep, self.state, 1) self.rc.encode_bit(self.p_is_rep_g0, self.state, 0) self.rc.encode_bit(self.p_rep0_long, (self.state << kNumPosBitsMax) + ps, 1) self._encode_len(self.p_rep_len_choice, self.p_rep_len_low, self.p_rep_len_mid, self.p_rep_len_high, length) self.state = 8 if self.state < kNumLitStates else 11 self.pos += length
def finish(self): return self.rc.finish()
# ============================================================
# Part 2: tar + xz 格式原语
# ============================================================
MAX_MATCH = 273 # LZMA1 单个 match 长度上限CHUNK = 65536 # LZMA2 单 chunk 解压上限PROPS_BYTE = 0x5D # lc=3, lp=0, pb=2SEED_NOTE = ("\nMayx's Blog!").encode("utf-8")
# xz 格式常量
XZ_MAGIC = b"\xFD7zXZ\x00"XZ_FOOTER_MAGIC = b"YZ"XZ_CHECK_CRC64 = 0x04LZMA2_FILTER_ID = 0x21CHECK_SIZE = 8 # CRC64 占 8 字节_XZ_FLAGS = bytes([0x00, XZ_CHECK_CRC64]) # Stream Flags: reserved + check_type
# tar 格式常量
TAR_BLOCK = 512
def round_up_512(n): """长度按 tar 块 512 字节向上取整。""" return (n + TAR_BLOCK - 1) // TAR_BLOCK * TAR_BLOCK
def crc32(b): return binascii.crc32(b) & 0xFFFFFFFF
# ---------- CRC64 (ECMA-182, reflected) ----------
_CRC64_POLY = 0xC96C5795D7870F42 # reflected ECMA-182_CRC64_INIT = 0xFFFFFFFFFFFFFFFF_CRC64_XOROUT = 0xFFFFFFFFFFFFFFFF_CRC64_NON_REFLECTED = 0x142F0E1EBA9EA3693 # 非反射多项式 (65 bit)_CRC64_MOD_BIT = 1 << 64 # x^64 对应的 bit
_liblzma = Nonetry: _lzma_name = ctypes.util.find_library('lzma') if _lzma_name: _liblzma = ctypes.CDLL(_lzma_name) _liblzma.lzma_crc64.argtypes = [ctypes.c_char_p, ctypes.c_size_t, ctypes.c_uint64] _liblzma.lzma_crc64.restype = ctypes.c_uint64 # 验证:CRC64("123456789") 应为 0x995dc9bbdf1939fa if _liblzma.lzma_crc64(b'123456789', 9, 0) != 0x995dc9bbdf1939fa: _liblzma = Noneexcept (OSError, AttributeError): _liblzma = None
def _make_crc64_table(): table = [] for i in range(256): crc = i for _ in range(8): crc = (crc >> 1) ^ _CRC64_POLY if (crc & 1) else crc >> 1 table.append(crc) return table
_CRC64_TABLE = _make_crc64_table()
def _crc64_pure(data): """纯 Python CRC64(ECMA-182 反射,init/xorout = 0xFFFF...FFFF)。""" crc = _CRC64_INIT for byte in data: crc = (crc >> 8) ^ _CRC64_TABLE[(crc ^ byte) & 0xFF] return crc ^ _CRC64_XOROUT
def _crc64_lib(data): """liblzma CRC64(init=0,内部处理 init/xorout)。""" return _liblzma.lzma_crc64(data, len(data), 0)
crc64 = _crc64_lib if _liblzma else _crc64_pure
# ---------- GF(2^64) 多项式运算 ----------
# 用于 CRC64 自引用求解。CRC64 的线性贡献可表示为
# bit_reverse(contribution(D)) = D_poly * P (mod G_non)
# 其中 D_poly = bit_reverse(D),P 是位置决定的多项式。
# 方程 D_poly * (1 ^ P1 ^ P2) = bit_reverse(CRC64(W)) 通过多项式逆元求解。
def _bit_reverse64(v): r = 0 for _ in range(64): r = (r << 1) | (v & 1) v >>= 1 return r
def _poly_mul_mod(a, b): """GF(2) 多项式乘法 mod G(非反射)。""" result = 0 while b: if b & 1: result ^= a b >>= 1 a <<= 1 if a & _CRC64_MOD_BIT: a ^= _CRC64_NON_REFLECTED return result
def _poly_pow(base, exp): """多项式幂:base^exp mod G。""" result = 1 while exp: if exp & 1: result = _poly_mul_mod(result, base) base = _poly_mul_mod(base, base) exp >>= 1 return result
def _poly_minv(p): """多项式逆元 p^(-1) mod G,用扩展欧几里得算法。""" if p == 0: return 0 old_r, r = p, _CRC64_NON_REFLECTED old_s, s = 1, 0 while r: # poly divmod(old_r, r) -> (q, rem) probe_r = r.bit_length() - 1 rem = old_r q = 0 while rem.bit_length() - 1 >= probe_r and rem: shift = rem.bit_length() - 1 - probe_r q |= 1 << shift rem ^= r << shift old_r, r = r, rem # s = old_s ^ q*s (raw 多项式乘法) prod = 0 a, b = q, s while b: if b & 1: prod ^= a b >>= 1 a <<= 1 old_s, s = s, old_s ^ prod if old_r != 1: return 0 # old_s mod G probe_m = _CRC64_NON_REFLECTED.bit_length() - 1 rem = old_s while rem.bit_length() - 1 >= probe_m and rem: shift = rem.bit_length() - 1 - probe_m rem ^= _CRC64_NON_REFLECTED << shift return rem
def store_hdr(payload_len, first=False): """LZMA2 uncompressed chunk 头(3 字节,大端 size-1)。""" assert 1 <= payload_len <= 65536 return bytes([0x01 if first else 0x02]) + (payload_len - 1).to_bytes(2, "big")
def matches_for(dist, total): """dist 固定、总长 total 的 token 序列:首个 match + 后续 rep0。""" assert total >= 2 toks = [] first = min(MAX_MATCH, total) if total - first == 1: first -= 1 toks.append(("m", dist, first)) total -= first while total > 0: l = min(MAX_MATCH, total) if total - l == 1: l -= 1 toks.append(("r0", l)) total -= l return toks
def _add_gear(chunks, o, f, q): """追加一组标准 gear:store 载荷 x = slip+3,紧跟一个 lzma 把那 x 字节复制一遍。
lzma chunk 解压出的字节数正好等于 x,所以齿轮转完之后 slip (f - (o-q)) 精确 归约为 len(cb)(很小的正数,x 被 CHUNK 截断时则按 CHUNK 缩减)。 _layout_iterate 反复套用把 slip 压到 <= 16,_trim_layouts 用它做相位微调。 返回追加后的 (o, f)。 """ x = min(f - (o - q) + 3, CHUNK) chunks.append({"kind": "store", "foff": f, "size": 3 + x, "ooff": o, "olen": x}) o += x f += 3 + x cb, cu = lzma_chunk(matches_for(x, x), o) chunks.append({"kind": "lzma", "foff": f, "size": len(cb), "ooff": o, "olen": cu, "bytes": cb}) return o + cu, f + len(cb)
def _forward_vac(F, vac, q): """vac_store:把紧随其后的等长字节块前移覆盖自身。""" fo = vac["ooff"] - q olen = vac["olen"] F[fo:fo + olen] = F[fo + olen:fo + 2 * olen]
def dict_prop_for(maxdist): """选最小的 LZMA2 dict prop 使字典 >= maxdist。""" for p in range(41): if (2 | (p & 1)) << (p // 2 + 11) >= maxdist: return p, (2 | (p & 1)) << (p // 2 + 11) raise ValueError("distance too large")
def lzma_chunk(tokens, out_pos): """编一个 LZMA chunk(0xC0: state+props reset,无 dict reset,无 end marker)。""" enc = LzmaEncoder() enc.pos = out_pos total = 0 for t in tokens: if t[0] == "m": enc.match(t[1], t[2]) total += t[2] else: enc.rep_match(t[1]) total += t[1] data = enc.finish() assert total - 1 < 65536 and len(data) - 1 < 65536 hdr = bytes([0xC0]) + (total - 1).to_bytes(2, "big") + \ (len(data) - 1).to_bytes(2, "big") + bytes([PROPS_BYTE]) return hdr + data, total
# ---------- tar 原语 ----------
def tar_header(name, size, mtime, mode=0o644, typeflag=b'0', uid=0, gid=0): """构建 512 字节 POSIX ustar tar header。 注意:tar 数值字段包含 null 终止符,slice 范围必须精确匹配字段宽度。 mode/uid/gid 各 8 字节(7 octal + NUL),size/mtime 各 12 字节(11 octal + NUL)。 """ h = bytearray(512) name_bytes = name.encode('utf-8')[:100] h[0:len(name_bytes)] = name_bytes h[100:108] = b'%07o\x00' % mode # 8 bytes (field 100-107) h[108:116] = b'%07o\x00' % uid # 8 bytes (field 108-115) h[116:124] = b'%07o\x00' % gid # 8 bytes (field 116-123) h[124:136] = b'%011o\x00' % size # 12 bytes (field 124-135) h[136:148] = b'%011o\x00' % mtime # 12 bytes (field 136-147) h[148:156] = b' ' # checksum placeholder (8 spaces) h[156:157] = typeflag h[257:263] = b'ustar\x00' h[263:265] = b'00' assert len(h) == 512, "header length %d != 512" % len(h) # 计算校验和:所有字节之和,checksum 字段视为空格 chksum = sum(h) & 0o7777777 h[148:156] = b'%06o\x00 ' % chksum # 6 octal + NUL + space = 8 bytes assert len(h) == 512, "header length %d != 512 after checksum" % len(h) return bytes(h)
# ---------- xz 原语 ----------
def xz_varint(v): """xz 变长整数编码:每字节 7 位数据(LSB first),MSB 为续位。""" b = [] while v >= 0x80: b.append((v & 0x7F) | 0x80) v >>= 7 b.append(v & 0x7F) return bytes(b)
def xz_stream_header(): """12 字节 xz stream header (CHECK_CRC64)。 Stream Flags = [reserved(0x00), check_type(CRC64=0x04)]。 """ flags = _XZ_FLAGS crc = crc32(flags) return XZ_MAGIC + flags + struct.pack('<I', crc)
def xz_block_header(dict_size): """xz block header(单 LZMA2 filter)。 返回 (header_bytes, 实际字典大小);header 长度即 block header 总长度。 """ prop, real_dict = dict_prop_for(dict_size) # filter flags: filter_id(1) + props_size(1) + props(1=prop byte) filter_flags = bytes([LZMA2_FILTER_ID, 1, prop]) # block header content: flags(1, 0x00=1 filter) + filter_flags content = bytes([0x00]) + filter_flags # 填充到 (1 + len(content) + 4) 是 4 的倍数 total = 1 + len(content) + 4 pad = (4 - total % 4) % 4 content += b'\x00' * pad bh = 1 + len(content) + 4 assert bh % 4 == 0, "bh not 4-aligned: %d" % bh crc = crc32(bytes([bh // 4 - 1]) + content) header = bytes([bh // 4 - 1]) + content + struct.pack('<I', crc) assert len(header) == bh return header, real_dict
def xz_index(records): """xz index。records = [(unpadded_size, uncompressed_size), ...]。""" data = bytes([0x00]) # index indicator data += xz_varint(len(records)) for unpadded, uncompressed in records: data += xz_varint(unpadded) data += xz_varint(uncompressed) # 填充到 (len(data) + 4) 是 4 的倍数 pad = (4 - (len(data) + 4) % 4) % 4 data += b'\x00' * pad return data + struct.pack('<I', crc32(data))
def xz_stream_footer(backward_size): """12 字节 xz stream footer (CHECK_CRC64)。 backward_size = (index_size / 4) - 1。 Stream Flags = [reserved(0x00), check_type(CRC64=0x04)]。 CRC32 覆盖 Backward Size + Stream Flags(6 字节,不含 CRC32 和 Footer Magic)。 """ flags = _XZ_FLAGS crc_data = struct.pack('<I', backward_size) + flags # 6 bytes, no magic crc = crc32(crc_data) return struct.pack('<I', crc) + crc_data + XZ_FOOTER_MAGIC
# ============================================================
# Part 3: 多文件 tar.xz quine 构造器
# ============================================================
class TarXzQuine: def __init__(self, root, quine_name="quine.tar.xz", seed_name=".this_is_mayx_blog", src_dir=""): self.quine_name = quine_name self.seed_name = seed_name self.src_dir = src_dir.strip("/") self.mtime = int(time.time()) # walk 源目录 self.walk_entries = [] self.files = [] if self.src_dir: parts = self.src_dir.split("/") for i in range(1, len(parts) + 1): self.walk_entries.append(("/".join(parts[:i]), True)) for dirpath, dirnames, filenames in os.walk(root): dirnames.sort() for dn in dirnames: rel = os.path.relpath(os.path.join(dirpath, dn), root).replace(os.sep, "/") self.walk_entries.append((self._prefixed(rel), True)) for fn in sorted(filenames): p = os.path.join(dirpath, fn) rel = os.path.relpath(p, root).replace(os.sep, "/") with open(p, "rb") as fp: data = fp.read() self.walk_entries.append((self._prefixed(rel), False)) self.files.append((self._prefixed(rel), data)) # rel -> data 映射,避免下面三处按 rel 回头线性搜索整个 files 列表 self.file_map = dict(self.files) self.dirs = [r for r, isd in self.walk_entries if isd] self.content = b"".join(data for _, data in self.files)
def _prefixed(self, rel): return self.src_dir + "/" + rel if self.src_dir else rel
def tar_entries(self, d_seed, xz_size): """计算 tar 归档中每个条目的 (header, data_size, is_dir)。 顺序:seed, walk_entries (interleaved files+dirs), quine。 """ entries = [] # seed 文件 seed_hdr = tar_header(self.seed_name, d_seed, self.mtime) entries.append({"name": self.seed_name, "hdr": seed_hdr, "data_size": d_seed, "is_dir": False}) # walk 顺序的文件和目录 for rel, isd in self.walk_entries: if isd: hdr = tar_header(rel, 0, self.mtime, mode=0o755, typeflag=b'5') entries.append({"name": rel, "hdr": hdr, "data_size": 0, "is_dir": True}) else: # file_map 与 walk_entries 在 __init__ 里同步构造,这里必有 data = self.file_map[rel] entries.append({"name": rel, "hdr": tar_header(rel, len(data), self.mtime), "data_size": len(data), "is_dir": False}) # quine 文件(放在最后) qhdr = tar_header(self.quine_name, xz_size, self.mtime) entries.append({"name": self.quine_name, "hdr": qhdr, "data_size": xz_size, "is_dir": False}) return entries
# ---------- 结构计算 ---------- def _compute_q(self, d_seed): """计算 xz_bytes 在 tar 中的偏移 q。 q = seed + files + dirs 的 tar 占用 + quine tar header。 q 不依赖 xz_size(quine 的 data 在 q 之后)。""" q = 0 # seed 文件 q += TAR_BLOCK # seed tar header q += round_up_512(d_seed) # seed data + pad # walk 顺序的文件和目录 for rel, isd in self.walk_entries: q += TAR_BLOCK # tar header if not isd: q += round_up_512(len(self.file_map[rel])) # quine tar header q += TAR_BLOCK return q
def _trim_layouts(self, base, o_g, f_g, q, w_plant_off, d_seed): """枚举 trim 配置,产出一批「相位」不同的候选布局。
trim = [可选的裸 store chunk,载荷长度 x0] + 一组标准 gear (store 载荷 x = s+3,紧接着一个 lzma 复制这 x 字节)。
裸 store 把 slip 抬高 3、把 f 抬高 3+x0;随后这组 gear 把 slip 重新归一到 len(cb),jump 再归一到 -3。净效果是 (o_pre, f_pre) 被整体平移 δ ≈ s_g + 6 + len(cb) + 3 + x0,而 δ 决定了自洽方程的模 4 相位。 旧实现 δ 恒为 0,相位一旦不对就永远无解 —— 而 d_seed += 1 根本不改变 布局(q 只在跨 512 字节块时才变),所以那 80 次重试是纯粹的无效空转。
注意:不能只插一个裸的 lzma 复制块。lzma 复制块的输出必须等于 F[o-q : o-q+olen],而这只能靠前面那个「载荷 = F[o-q : o-q+x] 且 x = s+3」 的 store chunk 把字节搬到正确位置来实现;裸插复制块解压出来和 F 对不上。
返回 [(chunks, o_before_jump, f_before_jump, y, jb), ...],按文件体积升序。 """ s_g = f_g - (o_g - q) out = [] seen = set() for x0 in range(0, s_g + 4): chunks = list(base) o, f = o_g, f_g if x0: # 裸 store:载荷 = F[o-q : o-q+x0],要求 x0 <= s+3 以免读到尚未定稿的字节 chunks.append({"kind": "store", "foff": f, "size": 3 + x0, "ooff": o, "olen": x0}) o += x0 f += 3 + x0 o, f = _add_gear(chunks, o, f, q) # 一组标准 gear # jump 不动点:y = s + 3 + len(jb) s = f - (o - q) y = max(2, s + 17) jb = ju = None for _ in range(64): try: jb, ju = lzma_chunk([("m", o - w_plant_off, y)], o) except AssertionError: jb = None break y2 = s + 3 + len(jb) if y2 == y: break if y2 < 2 or y2 > 273: jb = None break y = y2 if jb is None or ju != y: continue if y + w_plant_off - TAR_BLOCK > d_seed: continue key = (o + y, f + len(jb)) if key in seen: continue seen.add(key) out.append((chunks, o, f, y, jb))
out.sort(key=lambda t: t[2] + len(t[4])) return out
def _solve_gadget(self, chunks_pre, o_j, f_j, y, jb, q, bh, FEW, block_hdr, dict_size, hdrA, hdrB, d_seed, k): """在一个给定相位的布局上求解 gadget(T / gb / suffix / tz / k)。
关键简化:tz 只依赖 u = gb + suffix 与 k;T = u + tz + 4 也只依赖 (u, k); 而 gb 又由 T 唯一确定(len(lzma_chunk(matches_for(T, T))))。 因此只需一维枚举 u 即可遍历全部解,既不漏解也远快于原来的三重暴力。 """ o_pre = o_j + y f_pre = f_j + len(jb) if f_pre != (o_pre - q) - 3: # jump 应把 slip 归一到 -3 return None
chunks = list(chunks_pre) chunks.append({"kind": "lzma", "foff": f_j, "size": len(jb), "ooff": o_j, "olen": y, "bytes": jb, "jump": True})
best = None for u in range(8, 512): # 512k 必须落在 [f_pre + 2u + 2059, f_pre + 2u + 3595) lo = f_pre + 2 * u + 2059 hi = f_pre + 2 * u + 3595 for k_cand in range(max(1, (lo - 512) // 512), (hi + 512) // 512 + 2): num = k_cand * 512 - (o_pre + 2 * u + 8 - q - 1024) if num % 3 != 0: continue tz_cand = num // 3 if tz_cand < 1024: continue T = u + tz_cand + 4 if T < 2: continue gb, gu = lzma_chunk(matches_for(T, T), o_pre + T) gb_len = len(gb) suffix = u - gb_len if suffix < 1: continue
# 精确自检(不再用带 9 字节偏差的预筛公式) f_final = f_pre + 3 + T + gb_len + 3 + tz_cand + 1 n = f_final - (12 + bh) bp = (4 - (12 + bh + n) % 4) % 4 unpadded = bh + n + CHECK_SIZE idx = xz_index([(unpadded, q + k_cand * 512 + 1024)]) ix = len(idx) suffix_actual = bp + CHECK_SIZE + ix + 12 if suffix_actual != suffix: continue xz_size = 12 + bh + n + bp + CHECK_SIZE + ix + 12 xz_pad_actual = round_up_512(xz_size) if xz_pad_actual != k_cand * 512: continue total_w_actual = q + xz_pad_actual + 1024 o_final = o_pre + 2 * T + tz_cand if o_final != total_w_actual: continue
if best is not None and xz_size >= best["xz_size"]: continue # 已找到更小的解,跳过
backward_size = ix // 4 - 1 final = list(chunks) final.append({"kind": "store", "foff": f_pre, "size": 3 + T, "ooff": o_pre, "olen": T, "vac": True}) final.append({"kind": "lzma", "foff": f_pre + 3 + T, "size": gb_len, "ooff": o_pre + T, "olen": gu, "bytes": gb}) final.append({"kind": "store", "foff": f_pre + 3 + T + gb_len, "size": 3 + tz_cand, "ooff": o_pre + 2 * T, "olen": tz_cand, "trailing": True}) final.append({"kind": "term", "foff": f_pre + 3 + T + gb_len + 3 + tz_cand, "size": 1, "ooff": o_final, "olen": 0, "bytes": b"\x00"}) best = { "q": q, "d_seed": d_seed, "k": k, "chunks": final, "block_hdr": block_hdr, "bh": bh, "n": n, "xz_size": xz_size, "T": T, "suffix": suffix_actual, "bp": bp, "ix": ix, "dict_size": dict_size, "FEW": FEW, "hdrA": hdrA, "hdrB": hdrB, "jump_y": y, "index": idx, "stream_hdr": xz_stream_header(), "footer": xz_stream_footer(backward_size), "entries": self.tar_entries(d_seed, xz_size), } return best
def _layout_iterate(self, d_seed, maxdist_hint): """核心布局。
阶段 1a:C1 / repro / gears(只依赖 q,所有 trim 配置共用,只算一次) 阶段 1b:插入 trim 微调块,平移 (o_pre, f_pre) 改变模 4 相位 阶段 2 :按 u = gb + suffix 一维搜索自洽解 """ block_hdr, dict_size = xz_block_header(maxdist_hint) bh = len(block_hdr) FEW = 12 + bh + 3 w_hdrA_off = TAR_BLOCK + 12 + bh w_hdrB_off = TAR_BLOCK + 12 + bh + 3 w_plant_off = TAR_BLOCK + 12 + bh + 6
q = self._compute_q(d_seed)
# ---------- 阶段 1a:C1 + repro + gears ---------- base = [] o, f = 0, 12 + bh Ls = [] rem = q + FEW while rem > CHUNK: Ls.append(CHUNK) rem -= CHUNK Ls.append(rem) assert Ls[-1] >= 2, "末 C1 chunk 载荷 %d 太小" % Ls[-1] k = len(Ls) hdrA = store_hdr(CHUNK, first=False) hdrB = store_hdr(Ls[-1], first=False)
S = 0 for j, L in enumerate(Ls): base.append({"kind": "store", "foff": f, "size": 3 + L, "ooff": o, "olen": L, "first": j == 0, "c1": True, "S": S}) o += L f += 3 + L S += L assert o == q + FEW
S = 0 for j, L in enumerate(Ls): if j > 0: srcpos = w_hdrA_off if (j < k - 1 or L == CHUNK) else w_hdrB_off dist_h = (q + FEW + S + 3 * (j - 1)) - srcpos hb, hu = lzma_chunk([("m", dist_h, 3)], o) base.append({"kind": "lzma", "foff": f, "size": len(hb), "ooff": o, "olen": hu, "bytes": hb}) o += hu f += len(hb) cb, cu = lzma_chunk(matches_for(q + FEW + 3 * j, L), o) base.append({"kind": "lzma", "foff": f, "size": len(cb), "ooff": o, "olen": cu, "bytes": cb}) o += cu f += len(cb) S += L
# gears:把 slip 压到 <= 16 while f - (o - q) > 16: o, f = _add_gear(base, o, f, q)
o_g, f_g = o, f
# ---------- 阶段 1b + 阶段 2 ---------- # 相位按文件体积升序枚举;第一个成功的相位通常已经足够好,但不同相位的 # (u, tz) 不同,xz_size 仍可能相差几百字节,所以再多看几个取最小的。 tried = 0 best = None since_hit = 0 for chunks_pre, o_j, f_j, y, jb in self._trim_layouts( base, o_g, f_g, q, w_plant_off, d_seed): lay = self._solve_gadget(chunks_pre, o_j, f_j, y, jb, q, bh, FEW, block_hdr, dict_size, hdrA, hdrB, d_seed, k) tried += 1 if lay is not None and (best is None or lay["xz_size"] < best["xz_size"]): best = lay if best is not None: since_hit += 1 if since_hit >= 10: break elif tried >= 80: # 一直无解,放弃后续相位 break if best is not None: return best
raise RuntimeError("layout 不收敛: 无自洽解 (q=%d, f_pre=%d, o_pre=%d)" % (q, f_g, o_g))
# ---------- 装配 ---------- def assemble(self, lay): """构建 xz 文件 F 和 seed。""" F = bytearray(lay["xz_size"]) q = lay["q"] d_seed = lay["d_seed"] bh = lay["bh"] FEW = lay["FEW"]
# bytearray 已全零初始化:block padding、CRC64 占位、trailing-zero 载荷 # 都不必再显式写 0(后面几处赋值只会写到各自独立的区间,不会交叉覆盖)。
# pass A: 写 xz stream header + block header + 所有 chunk bytes + suffix F[0:12] = lay["stream_hdr"] F[12:12+bh] = lay["block_hdr"]
jump_c = None for c in lay["chunks"]: if c["kind"] == "store": F[c["foff"]:c["foff"] + 3] = store_hdr(c["olen"], first=c.get("first", False)) else: F[c["foff"]:c["foff"] + c["size"]] = c["bytes"] if c.get("jump"): jump_c = c
# block padding + CRC64(pass D 填) + index + footer bp_off = 12 + bh + lay["n"] ix_off = bp_off + lay["bp"] + CHECK_SIZE F[ix_off:ix_off + lay["ix"]] = lay["index"] sf_off = ix_off + lay["ix"] F[sf_off:sf_off + 12] = lay["footer"]
assert jump_c is not None
# pass B: 构建 seed y = lay["jump_y"] plant = bytes(F[jump_c["ooff"] - q: jump_c["ooff"] - q + y]) seed = bytearray(d_seed) seed[0:12] = lay["stream_hdr"] seed[12:12+bh] = lay["block_hdr"] seed[12+bh:12+bh+3] = lay["hdrA"] seed[12+bh+3:12+bh+6] = lay["hdrB"] seed[12+bh+6:12+bh+6+y] = plant pad = (SEED_NOTE * (d_seed // len(SEED_NOTE) + 2))[:d_seed - 12 - bh - 6 - y] seed[12+bh+6+y:] = pad
# 构建 tar 前缀 T(含 seed data) T = self._build_tar_prefix_with_seed(lay, seed) F_prefix = bytes(F[0:FEW]) P = T + F_prefix assert len(P) == q + FEW, "P 长度 %d != q+FEW %d" % (len(P), q + FEW)
# pass C: 填充 store payloads for c in lay["chunks"]: if c["kind"] != "store": continue ooff, olen = c["ooff"], c["olen"] if c.get("c1"): F[c["foff"] + 3:c["foff"] + 3 + olen] = P[c["S"]:c["S"] + olen] elif c.get("trailing"): # 载荷是尾部零:F 全零初始化,且唯一会写到 chunk 之外的 vac_store # 目标是 [f_pre+3, f_pre+3+T)(f_pre == (o_pre-q)-3 已由 _solve_gadget # 断言),与本区块不重叠,因此无需写入。 continue elif c.get("vac"): _forward_vac(F, c, q) else: # 其余 store 的载荷 = 输出流自身对应位置的字节(自引用复制) fo = ooff - q F[c["foff"] + 3:c["foff"] + 3 + olen] = F[fo:fo + olen]
# pass D: 求解 CRC64 自引用 self._solve_crc64(F, lay, T)
return bytes(F), bytes(seed)
def _build_tar_prefix_with_seed(self, lay, seed): """构建 tar 前缀 T(含 seed data)。""" entries = lay["entries"] T = bytearray() for e in entries[:-1]: T += e["hdr"] if not e["is_dir"] and e["data_size"] > 0: data = seed if e["name"] == self.seed_name else self.file_map[e["name"]] T += data T += b'\x00' * (round_up_512(e["data_size"]) - e["data_size"]) T += entries[-1]["hdr"] # quine tar header return bytes(T)
def _solve_crc64(self, F, lay, T_prefix): """求解 CRC64 自引用。
CRC64 值 D = CRC64(W),W = T + F + 尾部零。 D 在 W 中出现两次:F 的 CRC64 字段 + vac_store 副本。
解法(多项式逆元): CRC64 的线性贡献可表示为 GF(2^64) 多项式乘法: bit_reverse(contribution(D)) = D_poly * P (mod G) 其中 D_poly = bit_reverse(D),P = x^(64+trailing) mod G。 方程 D_poly * (1 ^ P1 ^ P2) = bit_reverse(CRC64(W)) (mod G) 用扩展欧几里得求 (1 ^ P1 ^ P2) 的逆元,一步求解。 """ q = lay["q"] bh = lay["bh"] n = lay["n"] bp = lay["bp"] xz_size = lay["xz_size"] T = lay["T"]
# CRC64 位置 1:在 F 中(block padding 后) crc_f_off = 12 + bh + n + bp crc_w1 = q + crc_f_off
# CRC64 位置 2:vac_store 副本 crc_f_copy = crc_f_off - T crc_w2 = q + crc_f_copy
# 构造 W = T + F + 尾部零(先把两处 CRC64 置 0) xz_pad = round_up_512(xz_size) tz = xz_pad - xz_size + 1024 W = bytearray(T_prefix) + F + b'\x00' * tz W[crc_w1:crc_w1 + CHECK_SIZE] = b'\x00' * CHECK_SIZE W[crc_w2:crc_w2 + CHECK_SIZE] = b'\x00' * CHECK_SIZE
# CRC64(W) 的多项式表示 crc_poly = _bit_reverse64(crc64(bytes(W)))
# P1 = x^(64 + 位置1之后的bit数) mod G trailing1 = (len(W) - crc_w1 - 8) * 8 P1 = _poly_pow(2, 64 + trailing1) # P2 = x^(64 + 位置2之后的bit数) mod G trailing2 = (len(W) - crc_w2 - 8) * 8 P2 = _poly_pow(2, 64 + trailing2)
# D_poly = crc_poly * inv(1 ^ P1 ^ P2) coeff = 1 ^ P1 ^ P2 inv = _poly_minv(coeff) if inv == 0: raise RuntimeError("CRC64 多项式无逆元") D_poly = _poly_mul_mod(crc_poly, inv) D = _bit_reverse64(D_poly)
# 写入 D 到 F 的 CRC64 位置 struct.pack_into('<Q', F, crc_f_off, D)
# 重新执行 vac_store 副本 _forward_vac(F, next(c for c in lay["chunks"] if c.get("vac")), q)
# 验证 W2 = bytearray(T_prefix) + F + b'\x00' * tz assert crc64(bytes(W2)) == D, "CRC64 求解验证失败"
def build(self, maxdist_hint=None): if maxdist_hint is None: maxdist_hint = 2 * (len(self.content) + 128) + (1 << 21) last_err = None # d_seed 在同一 512 字节块内不改变 q(seed 数据按 512 对齐),所以旧的 # "d_seed += 1" 重试完全不会改变布局,只是空转 80 次。真正需要 d_seed # 变化的情况只有两种:seed 装不下 jump 种植串,或需要跨块改变 q 作为兜底。 for d_seed in (64, 96, 128, 160, 192, 256, 320, 384, 448, 576, 704, 832, 960, 1088, 1600, 2112, 2624, 3648): try: lay = self._layout_iterate(d_seed, maxdist_hint) F, seed = self.assemble(lay) return F, lay, seed except (RuntimeError, AssertionError, ValueError) as e: last_err = e raise RuntimeError("build 多次失败: %s" % last_err)
# ============================================================
# Part 4: 验证 + main
# ============================================================
def verify(F, lay, files, quine_name): """验证:解压 xz → 解析 tar → 检查文件 + quine 自复制。""" # 1. 解压 xz try: dec = lzma.LZMADecompressor(format=lzma.FORMAT_XZ) tar_data = dec.decompress(F) except Exception as e: print("[verify] xz 解压失败: %s" % e) return False
# 2. 检查 quine 自复制 if tar_data[lay["q"]:lay["q"] + lay["xz_size"]] != F: print("[verify] quine 自复制不匹配") a = tar_data[lay["q"]:lay["q"] + lay["xz_size"]] for i in range(min(len(a), len(F))): if a[i] != F[i]: print("[verify] 首个不匹配在偏移 %d: tar=%02x vs xz=%02x" % (i, a[i], F[i])) break return False
# 3. 解析 tar try: tf = tarfile.open(fileobj=io.BytesIO(tar_data)) members = tf.getmembers() except Exception as e: print("[verify] tar 解析失败: %s" % e) return False
print("[verify] tar 条目: %s" % [m.name for m in members])
# 4. 检查源文件 ok = True for rel, data in files: try: f = tf.extractfile(rel) if f is None: print("[verify] 文件 %s 是目录或不存在" % rel) ok = False continue extracted = f.read() if extracted != data: print("[verify] 文件 %s 内容不匹配" % rel) ok = False except KeyError: print("[verify] 文件 %s 不在 tar 中" % rel) ok = False
# 5. 检查 quine 文件 try: qf = tf.extractfile(quine_name) if qf: qdata = qf.read() if qdata != F: print("[verify] quine 文件内容不匹配 (%d vs %d)" % (len(qdata), len(F))) ok = False else: print("[verify] quine 文件匹配!") except KeyError: print("[verify] quine 文件不在 tar 中") ok = False
print("[verify] 结果: %s" % ("通过" if ok else "失败")) return ok
def main(): ap = argparse.ArgumentParser(description="多文件 tar.xz quine 生成器") ap.add_argument("srcdir") ap.add_argument("output") ap.add_argument("--quine-name", default=None, help="归档内 quine 文件名(默认 = output 的文件名)") ap.add_argument("--seed-name", default=".this_is_mayx_blog") ap.add_argument("--src-dir", default="", help="源文件在归档内的前缀目录") ap.add_argument("--quine-dir", default="", help="quine 在归档内所在目录") args = ap.parse_args()
quine_name = args.quine_name or os.path.basename(args.output) if args.quine_dir: quine_dir = args.quine_dir.strip("/") quine_name = quine_dir + "/" + quine_name
bq = TarXzQuine(args.srcdir, quine_name=quine_name, seed_name=args.seed_name, src_dir=args.src_dir) print("[blogquine-tarxz] %d 个文件, %d 个目录, 内容 %d 字节" % (len(bq.files), len(bq.dirs), len(bq.content))) F, lay, seed = bq.build() with open(args.output, "wb") as fp: fp.write(F) print("[blogquine-tarxz] written %s: %d 字节 (q=%d, k=%d chunks, n=%d, xz_size=%d)" % (args.output, len(F), lay["q"], lay["k"], lay["n"], lay["xz_size"])) print("[blogquine-tarxz] T=%d, suffix=%d (bp=%d, ix=%d), dict_size=%d" % (lay["T"], lay["suffix"], lay["bp"], lay["ix"], lay["dict_size"])) ok = verify(F, lay, bq.files, quine_name) sys.exit(0 if ok else 1)
if __name__ == "__main__": main()```</details>
不过对我来说,相比于TXZ格式我还是更喜欢7z一点,一是因为LZMA2本来就是7-Zip的作者发明的,XZ感觉像是摘桃子的,二是TXZ这个名字听起来有点怪,感觉不像压缩包,三是XZ Utils出过后门,尽管整个Linux社区都在使用XZ,但是我还是稍微有点偏见,所以这份TXZ的生成器我就粘贴出来给需要的人吧,我博客用7z格式就好了。
# 感想
以前总是有人说AI没有创新能力,只是对曾经在网络上出现的东西进行重组,但人何尝不是这样呢?像这次制作的7z/TXZ Quine生成器在整个网络上没有任何公开信息,当然我也知道这并不是理论上的创新,但谁说组合创新不是创新呢?再看看最近OpenAI又解决了一大堆数学难题,完全可以相信AI是真的拥有智能,所以我相信总有一天AI将能完成人类能做的所有事情,人类将不再需要额外的思考,只需要做自己想做的事情吧。
|