TransWikia.com

Blktrace missing I/O events

Super User Asked on December 18, 2021

I’m using blktrace to capture the traffic that goes to my device and have been dealing with some painful issues. Wondering if you guys can confirm/correct my findings.

Seems like blktrace is missing some events. I’ve been trying to get distribution of latency for read commands issued to the device. I know blkparse and btt do that. However upon closer inspection of blkparse results, it seems that the latency numbers are actually response to response (C to C) latencies instead of command to response (D to C). BTT also fails as it encounters commands that don’t have responses (D is there but C is missing).

So I tried developing my own code and realized that there are many cases of commands issued with no response found (D is there but no C for it). Also many cases of having a response without a command for it (C is there but no prior D seen).
To confirm my finding I ran FIO on a nullblk device (uses DRAM so it’s pretty fast) and enabled blktrace to track the nullblk device. I see that when FIO logs 80GB of data write, blktrace captures only around 50GB of data to the disk.
I’m kind of confused and not sure about this conclusion but it seems that blktrace in heavy workloads drops events.

Is that something that you guys have noticed too?
Can someone point me to my mis-analysis or confirm my finding?

The fio command I used is this:

sudo fio --name=seqwrite --ioengine=libaio --iodepth=1 --rw=randwrite --bs=128k --direct=0 --size=130M --numjobs=600 --runtime=300 --group_reporting

One Answer

Unfortunately you didn't include your blktrace summary output - does it mention a non-zero number of dropped events? If so, the likelihood is the ring buffer that holds events until they can be read overflowed. If that's the case you may be able to workaround the issue by asking for a bigger buffer using the --buffer-size option mentioned in man page...

Answered by Anon on December 18, 2021

Add your own answers!

Ask a Question

Get help from others!

© 2024 TransWikia.com. All rights reserved. Sites we Love: PCI Database, UKBizDB, Menu Kuliner, Sharing RPP