LinuxQuestions.org - [SOLVED] Systemd fails to write

- Linux - Newbie (https://www.linuxquestions.org/questions/linux-newbie-8/)

- - Systemd fails to write (https://www.linuxquestions.org/questions/linux-newbie-8/systemd-fails-to-write-4175527467/)

Systemd fails to write

Hi...
See the screenshot...it ways...

Quote:

systemd-journald[113]:failed to write entry(21 items, 518bytes) ignoring: cannot assign requested address

Anything I can do? Anything I did wrong?
Pleae help, thanks
Thor

What do 'journalctl -xn' and 'dmesg' say?

hi unSpawn, thanks for being there :)

journalctl -xn

Quote:

Dec 05 23:56:47 Ning org.gtk.Private.UDisks2VolumeMonitor[973]: index_parse.c:191: indx_parse(): error opening /run/media/thor/5898-4FF1/BDMV/index.bdmv
Dec 05 23:56:47 Ning org.gtk.Private.UDisks2VolumeMonitor[973]: index_parse.c:191: indx_parse(): error opening /run/media/thor/5898-4FF1/BDMV/BACKUP/index.bdmv
Dec 06 00:40:09 Ning su[3294]: (to root) thor on pts/1
Dec 06 00:40:09 Ning su[3294]: pam_unix(su-l:session): session opened for user root by thor(uid=1000)
Dec 06 00:40:29 Ning su[3294]: pam_unix(su-l:session): session closed for user root
Dec 06 00:45:26 Ning org.freedesktop.thumbnails.Cache1[973]: (tumblerd:3341): tumbler-WARNING **: Failed to load plugin "tumbler-ffmpeg-thumbnailer.so": libffmpegthumbnailer.so.4: cann
Dec 06 00:45:27 Ning org.freedesktop.thumbnails.Cache1[973]: (tumblerd:3341): tumbler-WARNING **: Failed to load plugin "tumbler-odf-thumbnailer.so": libgsf-1.so.114: cannot open share
Dec 06 00:45:27 Ning org.freedesktop.thumbnails.Cache1[973]: (tumblerd:3341): tumbler-WARNING **: Failed to load plugin "tumbler-raw-thumbnailer.so": libopenrawgnome.so.1: cannot open
Dec 06 00:52:40 Ning su[3412]: (to root) thor on pts/2
Dec 06 00:52:40 Ning su[3412]: pam_unix(su-l:session): session opened for user root by thor(uid=1000)

dmesg
see text file :)

edit - I have a screenshot - taken with a camera...if ypu need to see it...
edit - I did not change any configs...
edit - my SL viewer crashed some times, long core dump...it that the problem?
edit - at shut down, it cannot write to persistent storage...
edit - one folder in /var/log/journal is 2.2 Gb big...

Thaks a zillion...
Thor
By the way - I'm making a backup since I saw this...that should explain the hi-speed USB device

Thanks for the logs. The first warning is at 10.233188 into the boot process about systemd-journald not able to write an entry. Just a hunch but it could be (surprise) something with your journal files. What does 'journalctl --verify' return? (While you're at it run 'journalctl --disk-usage' to see if the amount of disk space it uses is large or small enough for you.)

Hi, and goodmorning :)
I noticed a few things in the /var/log/journal
- the whole thing is beyond the 10 percent allowed, 2.2Gb on a filesystem of 20Gb
- some files end with .journal~ and are hidden
and the SystemMaxUse in /etc/systemd/journal.conf is commented out, it stands to reason systemd will symply eat everything...and not do any housekeeping...
Running this now, returning with the screendumps... :)
Thanks
Thor
Edit - the screendums

Quote:

PASS: /run/log/journal/555c265115ba4638865e93e4428cc4ca/system.journal
PASS: /var/log/journal/555c265115ba4638865e93e4428cc4ca/user-1000@000507616545c4c3-5aa12b10417e40c0.journal~
PASS: /var/log/journal/555c265115ba4638865e93e4428cc4ca/system@1b63605187944ef9a4f26bedd2baeba3-000000000002e51d-0005089e502b0267.journal
PASS: /var/log/journal/555c265115ba4638865e93e4428cc4ca/system@243ae9350e2f446f903ab507551e1910-00000000006ef2a1-000506580907899a.journal
PASS: /var/log/journal/555c265115ba4638865e93e4428cc4ca/system@243ae9350e2f446f903ab507551e1910-0000000000554a86-0005026be653306a.journal
PASS: /var/log/journal/555c265115ba4638865e93e4428cc4ca/system@243ae9350e2f446f903ab507551e1910-000000000064058c-000504b335c40e65.journal
PASS: /var/log/journal/555c265115ba4638865e93e4428cc4ca/user-1000@00050785d82d0452-67e433c66b85250a.journal~
PASS: /var/log/journal/555c265115ba4638865e93e4428cc4ca/system@243ae9350e2f446f903ab507551e1910-000000000060457b-000504112af2a514.journal
PASS: /var/log/journal/555c265115ba4638865e93e4428cc4ca/system@243ae9350e2f446f903ab507551e1910-00000000006ad34e-000505b8e685ac5e.journal
PASS: /var/log/journal/555c265115ba4638865e93e4428cc4ca/user-1000@f676117185e548c8ad5f5cb3b3294c86-00000000005cec25-000503a4a767afab.journal
PASS: /var/log/journal/555c265115ba4638865e93e4428cc4ca/user-1000@f676117185e548c8ad5f5cb3b3294c86-00000000006ad102-000505b8e4f004aa.journal
PASS: /var/log/journal/555c265115ba4638865e93e4428cc4ca/system@0005085a7fb821ca-15a15a2ffa1f1837.journal~
PASS: /var/log/journal/555c265115ba4638865e93e4428cc4ca/user-1000@0005085a80dbc808-eb50d5edd3ec62a7.journal~
PASS: /var/log/journal/555c265115ba4638865e93e4428cc4ca/system@00050785d41f869a-c62af9f3e5f13a73.journal~
PASS: /var/log/journal/555c265115ba4638865e93e4428cc4ca/user-1000@479e3ccacfa44612b4cec855abd58e6e-0000000000000468-0005085a80dfe1c2.journal
3b9678: LZ4 compressed object in file without LZ4 compression
File corruption detected at /var/log/journal/555c265115ba4638865e93e4428cc4ca/user-1000.journal:3b9678 (of 58720256 bytes, 6%).
FAIL: /var/log/journal/555c265115ba4638865e93e4428cc4ca/user-1000.journal (Bad message)
PASS: /var/log/journal/555c265115ba4638865e93e4428cc4ca/user-1000@f676117185e548c8ad5f5cb3b3294c86-000000000067a20b-0005058da70aff48.journal
PASS: /var/log/journal/555c265115ba4638865e93e4428cc4ca/system@1b63605187944ef9a4f26bedd2baeba3-0000000000000001-0005085a7f63ca5f.journal
PASS: /var/log/journal/555c265115ba4638865e93e4428cc4ca/system@243ae9350e2f446f903ab507551e1910-000000000067a4d7-0005058da7719d15.journal
PASS: /var/log/journal/555c265115ba4638865e93e4428cc4ca/system@243ae9350e2f446f903ab507551e1910-000000000058fdb2-000502f8464f32ce.journal
39ce70: invalid entry item (21/23 offset: 000000
39ce70: invalid object contents: Bad message
File corruption detected at /var/log/journal/555c265115ba4638865e93e4428cc4ca/system@0005076163e0d88b-ef9596b01c00fdbd.journal~:39ce70 (of 8388608 bytes, 45%).
FAIL: /var/log/journal/555c265115ba4638865e93e4428cc4ca/system@0005076163e0d88b-ef9596b01c00fdbd.journal~ (Bad message)
PASS: /var/log/journal/555c265115ba4638865e93e4428cc4ca/system@243ae9350e2f446f903ab507551e1910-00000000005cf3b9-000503a4ab644f04.journal
PASS: /var/log/journal/555c265115ba4638865e93e4428cc4ca/user-1000@479e3ccacfa44612b4cec855abd58e6e-00000000000337b3-0005089fd1684036.journal
PASS: /var/log/journal/555c265115ba4638865e93e4428cc4ca/user-1000@f676117185e548c8ad5f5cb3b3294c86-000000000063ffef-000504b32dc9a3e0.journal
PASS: /var/log/journal/555c265115ba4638865e93e4428cc4ca/user-1000@f676117185e548c8ad5f5cb3b3294c86-0000000000603fa2-0005041127e49bce.journal
PASS: /var/log/journal/555c265115ba4638865e93e4428cc4ca/user-1000@f676117185e548c8ad5f5cb3b3294c86-00000000006ed98b-0005065780b35783.journal
PASS: /var/log/journal/555c265115ba4638865e93e4428cc4ca/user-1000@f676117185e548c8ad5f5cb3b3294c86-000000000058fa9c-000502f844d037a7.journal
2e3ef80: LZ4 compressed object in file without LZ4 compression
File corruption detected at /var/log/journal/555c265115ba4638865e93e4428cc4ca/system.journal:2e3ef80 (of 67108864 bytes, 72%).
FAIL: /var/log/journal/555c265115ba4638865e93e4428cc4ca/system.journal (Bad message)

and the disk usage

Quote:

Journals take up 2.0G on disk.

Quote:

Originally Posted by Thor_2.0 (Post 5280369)

I noticed a few things in the /var/log/journal
- the whole thing is beyond the 10 percent allowed, 2.2Gb on a filesystem of 20Gb
- some files end with .journal~ and are hidden
and the SystemMaxUse in /etc/systemd/journal.conf is commented out, it stands to reason systemd will symply eat everything...and not do any housekeeping...

No idea why some files are hidden, probably a good reason for it somwhere deep, deep down in the documentation... 'man journald.conf' says SystemMaxUse defaults to 10% of the size of the respective file system. What you could do is move corrupt (or empty, it will throw a similar warning for that too) journal files out of the way, restart journald with 'systemctl kill --kill-who=main --signal=SIGUSR2 systemd-journald.service' and see if that helps.

owkay :)
do I need a system restart? Guess not...
Diving in :)
edit - oookay, the command
systemctl kill --kill-who=main --signal=SIGUSR2 systemd-journald.service
did...something, but it's a quiet one this one. No feedback...
so, ehm, I "do" something now to see if any messages appear...
disk usage is down to 1.9 Gb, the filesystem has 20Gb available... 10%....used as log...
edit - re runnign the verify, I could have missed a file...
edit - jup, missed one... :banghead:
edit - no go...one line per started program...
This - from the Arch (Manjaro's Mom) forum https://bbs.archlinux.org/viewtopic.php?id=158510
but...is it really safe to just go on and delete these logs?

Quote:

Originally Posted by Thor_2.0 (Post 5280378)

the command systemctl kill --kill-who=main --signal=SIGUSR2 systemd-journald.service did...something, but it's a quiet one this one. No feedback...

Indeed it doesn't but 'systemctl status systemd-journald.service' should tell you if all is OK.

Quote:

Originally Posted by Thor_2.0 (Post 5280378)

is it really safe to just go on and delete these logs?

Advise is just that and in the end you're responsible for what you do. Personally I'd rather not have to unfsck things and on top of that being careful pays anyway. So that's why I suggested you move corrupt journal files out of the way: you can always delete them when you have verified things are OK.

hmmpf, my though exactly, just deleting something is looking for doom...hehe, the statusreport:

Quote:

● systemd-journald.service - Journal Service
Loaded: loaded (/usr/lib/systemd/system/systemd-journald.service; static)
Active: active (running) since Sat 2014-12-06 08:37:58 CET; 4h 28min ago
Docs: man:systemd-journald.service(8)
man:journald.conf(5)
Main PID: 110 (systemd-journal)
Status: "Processing requests..."
CGroup: /system.slice/systemd-journald.service
└─110 /usr/lib/systemd/systemd-journald

Dec 06 08:37:51 Ning systemd-journal[110]: Runtime journal is using 8.0M (max allowed 151.5M, trying to leave 227.3M free of 1.4G available → current limit 151.5M).
Dec 06 08:37:51 Ning systemd-journal[110]: Permanent journal is using 2.0G (max allowed 1.9G, trying to leave 2.9G free of 10.5G available → current limit 2.0G).
Warning: Journal has been rotated since unit was started. Log output is incomplete or unavailable.

hmm, it seems she's out of space to put her stuff...poor gal...
(yea, I see my PC as a girl: misterious, compelling and a challenge)

Solved!
For prosperity, the good women and the beer, I hereby record the following
- open the console!
- get to be root
- examine the output of "journalctl -xn" and "dmesg"
- if you get this or something similar

Quote:

[ 12.713167] systemd-journald[113]: Failed to write entry (9 items, 245 bytes), ignoring: Cannot assign requested address

- verify the journals with "journalctl --verify"
- check if the logs are within sizes "journalctl --disk-usage"
- by then you have an idea of the corrupted logs - they show up iin red, no way you can ignore them
- open MC (yes THE MC :) )
- make a folder to MOVE the corrupted files to
- do a re run of the verify to make sure
- reboot

unSpawn, I owe you a beer!

Thanks...a zillion :)

Thor

Thanks for posting, that's thanks enough. BTW with the amount of systems I'm responsible for I'd rather not get into anthropomorphisation :-]