合 systemd-journald Failed to open runtime journal No space left on device
Tags： 故障处理LinuxNo space left on devicesystemd-journald
👉 本文共约2334个字，系统预计阅读时间或需9分钟。
目录
现象
通过Cockpit观察到日志报错
systemd-journald[606]: Failed to open runtime journal: No space left on device
Failed to write entry (9 items, 251 bytes) despite vacuuming, ignoring: No space left on device
/usr/lib/systemd/systemd-sysctl --prefix=/proc/sys/net/bridge' failed: Cannot allocate memory
日志其实就是/var/log/messages
分析
SUSE解决：
1 2 | touch /etc/machine-id systemd-machine-id-setup |
systemd-journald配置文件：
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 | [root@lhrdb log]# cat /etc/systemd/journald.conf # This file is part of systemd. # # systemd is free software; you can redistribute it and/or modify it # under the terms of the GNU Lesser General Public License as published by # the Free Software Foundation; either version 2.1 of the License, or # (at your option) any later version. # # Entries in this file show the compile time defaults. # You can change settings by editing this file. # Defaults can be restored by simply deleting this file. # # See journald.conf(5) for details. [Journal] #Storage=auto #Compress=yes #Seal=yes #SplitMode=uid #SyncIntervalSec=5m #RateLimitInterval=30s #RateLimitBurst=1000 #SystemMaxUse= #SystemKeepFree= #SystemMaxFileSize= #RuntimeMaxUse= #RuntimeKeepFree= #RuntimeMaxFileSize= #MaxRetentionSec= #MaxFileSec=1month #ForwardToSyslog=yes #ForwardToKMsg=no #ForwardToConsole=no #ForwardToWall=yes #TTYPath=/dev/console #MaxLevelStore=debug #MaxLevelSyslog=debug #MaxLevelKMsg=notice #MaxLevelConsole=info #MaxLevelWall=emerg #LineMax=48K [root@lhrdb log]# |
查看：
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 | [root@lhrdb log]# journalctl --list-boot 0 c0d6f93766a946da8c219f422abe568b Mon 2023-03-20 13:59:42 CST—Mon 2023-04-10 13:47:01 CST [root@lhrdb log]# journalctl --disk-usage Archived and active journals take up 200.0M on disk. [root@lhrdb log]# [root@lhrdb log]# systemctl status systemd-journald ● systemd-journald.service - Journal Service Loaded: loaded (/usr/lib/systemd/system/systemd-journald.service; static; vendor preset: disabled) Active: active (running) since Mon 2023-07-03 16:30:39 CST; 2 weeks 2 days ago Docs: man:systemd-journald.service(8) man:journald.conf(5) Main PID: 718 (systemd-journal) Status: "Processing requests..." Tasks: 1 Memory: 161.7M CGroup: /system.slice/systemd-journald.service └─718 /usr/lib/systemd/systemd-journald Jul 03 16:30:39 lhrdb systemd-journal[718]: Runtime journal is using 8.0M (max allowed 3.1G, trying to leave 4.0G free of 31.3G available → current limit 3.1G). Jul 03 16:30:39 lhrdb systemd-journal[718]: Journal started Jul 20 12:34:19 lhrdb systemd-journal[718]: Suppressed 8406 messages from /system.slice/docker.service [root@lhrdb log]# -- 修改/etc/systemd/journald.conf SystemMaxUse=100G RuntimeMaxUse=100G MaxRetentionSec=1month [root@lhrdb log]# systemctl restart systemd-journald [root@lhrdb log]# systemctl status systemd-journald ● systemd-journald.service - Journal Service Loaded: loaded (/usr/lib/systemd/system/systemd-journald.service; static; vendor preset: disabled) Active: active (running) since Thu 2023-07-20 13:57:36 CST; 5s ago Docs: man:systemd-journald.service(8) man:journald.conf(5) Main PID: 1410406 (systemd-journal) Status: "Processing requests..." Tasks: 1 Memory: 400.0K CGroup: /system.slice/systemd-journald.service └─1410406 /usr/lib/systemd/systemd-journald Jul 20 13:57:36 lhrdb systemd-journal[1410406]: Runtime journal is using 168.0M (max allowed 100.0G, trying to leave 4.0G free of 63.8G available → current limit 59.8G). Jul 20 13:57:36 lhrdb systemd-journal[1410406]: Journal started [root@lhrdb log]# |
报错 systemd-journald[13720]: Failed to write entry (19 items, 545 bytes), ignoring: Cannot assign requested address
从这个错误来看就是，journald写日志错误。
通过 journalctl --verify 命令找到损坏的journal文件，然后删除或者mv移动它，再重启systemd-journald服务即可。
参考文档：
https://www.claudiokuenzler.com/blog/627/systemd-journald-stopped-logging-centos-7-failed-to-write-entry#.Wn2G74KYPOQ
https://bugzilla.redhat.com/show_bug.cgi?id=1292447
解决
多半是run目录空间不足。Check that again when the problem exists.
1 2 3 4 5 | [root@lhrdb log]# df -h | grep run tmpfs 32G 204M 32G 1% /run tmpfs 6.3G 4.0K 6.3G 1% /run/user/988 tmpfs 6.3G 4.0K 6.3G 1% /run/user/0 [root@lhrdb log]# |
systemd-journald是什么服务
systemd-journald是一个Linux系统上的服务，它是systemd的一部分。systemd是一种系统和服务管理器，用于启动、管理和监控系统中的进程和服务。
本人提供Oracle(OCP、OCM)、MySQL(OCP)、PostgreSQL(PGCA、PGCE、PGCM)等数据库的培训和考证业务，私聊QQ646634621或微信dbaup66，谢谢！
