TransWikia.com

Arithmetic on values with memory size units

Unix & Linux Asked by Maëlan on December 20, 2020

Let’s say I have a bunch of numbers representing quantities of memory, written in the form 86k or 320m or 1.7g for instance. How can I compute their sum in command line, and get back a human-readable result?

Being able to compute subtractions would be nice too. The perfect tool would handle several sets of notations (such as 1g / 1G / 1GB / 1Go / 1GiB / 1.7Gio) and their meaning (binary or decimal multipliers).

I am looking for a pure calculator. These numbers are not necessarily the size of some files on my disk, so tools such as find, stat or du are not an option.

This is obviously easy to implement (with some hurdles regarding precision), but I would be damned if this didn’t exist already!

4 Answers

A little self promotion: we wrote a library called libbytesize to do these calculations in C and Python and it also has a commandline tool called bscalc

$ bscalc "5 * (100 GiB + 80 MiB) + 2 * (300 GiB + 15 GiB + 800 MiB)"
1215425413120 B
1186938880.00 KiB
   1159120.00 MiB
      1131.95 GiB
         1.11 TiB

The library is packaged in most distributions, unfortunately the tool isn't. It's in Fedora in libbytesize-tools and SuSE in bscalc package, but not in Debian/Ubuntu.

Correct answer by Vojtech Trefny on December 20, 2020

(it's one of the shells for *nix for those of you who are wondering) has built-in "SI" suffixes for numeric literals so you can use them directly

PS /home> $f = 5 * (100GB + 80MB) + 2 * (300GB + 15GB + 800MB)
PS /home> $f
1215425413120
PS /home> "In MB: " + $f/1MB
In MB: 1159120
PS /home> "In GB: "; $f/1GB
In GB:
1131.953125
PS /home> "In TB: {0}" -f ($f/1TB)
In TB: 1.10542297363281

Those are binary prefixes and not decimal prefixes so if you want decimal units just multiply by 1eX to get the desired value

PS /home> 1KB; 1MB; 1GB; 1TB; 1PB
1024
1048576
1073741824
1099511627776
1125899906842624
PS /home> $K = 1e3; $M = 1e6; $G = 1e9
PS /home> 1*$G
1000000000
PS /home> 5 * (100*$G + 80*$M) + 2 * (300*$G + 15*$G + 800*$M)
1132000000000

You can store those constants in the profile (similar to .bashrc in bash) to reuse them every time you open PowerShell

PowerShell is actually much more powerful than that. It runs on the .NET framework so it can do anything .NET can do: bigint math, decimal math, bitwise operations, trigonometry, date-time calculations...


Since the OP wants a pure calculator, I'll show some more examples on how PowerShell can be used for that purpose. The math functions above are mainly from the .NET Math class and Numerics namespace. You'll put class .NET types inside [], like [math] or [system.math] for the Math class (PowerShell is case-insensitive). Here are some other things that may be useful to programmers:

  • Bitwise operations (bitwise operators begin with -b except shift operators)

    [uint64]::MaxValue/3 + (-bnot 20) + (1L -shl 22) + (0x23 -band 0x34)
    
  • Big integer math: [bigint]::Pow([uint64]::MaxValue, 20)

  • Arbitrary integer and floating-point math expressions

    1.56 + 0.23/[math]::Pow([math]::Sqrt([math]::Log(20) + [math]::Sin([math]::PI/3)), 4)
    
  • Math on decimal type (128-bit): 1.23d * 3.45d / 28

  • Calculate file or object sizes: Use number suffixes 12.5GB + 5.8MB + 1392KB for binary units and 12.5e9 + 5.8e6 + 1392e3 for decimal units (G = 1e9, M = 1e6, K = 1e3)

  • Convert to/from base64: [Convert]::ToBase64String and [Convert]::FromBase64String

  • Date/time manipulation. For example convert from raw Epoch values to datetime and vice versa

      [datetime]::FromFileTime(0x01d15614cbaee92c)
      [datetime]::ParseExact("08-12-2012","dd-MM-yyyy", `
              [Globalization.CultureInfo]::InvariantCulture) 
    
  • String formatting and base conversion. Anything that String.Format in .NET supports will work. For more information read about the formatting operator. You can also do advanced string and regex manipulation. Some examples:

      'somestring'.Substring(4) * 3 -replace 'ings', 'eet'
      '{0:X}' -f (0x12 + 34)
      [convert]::ToString(0x12 + 34, 16)
      'This is an emoji' + [char]::ConvertFromUtf32(0x1F60A)
    
  • Direct XML and JSON manipulation

  • Call functions in *.SO (*.DLL in Windows) files directly

  • GUI programming. Here's a small sample clipboard history app

PowerShell sample

For more information read

or follow Dr Scripto's blog

Answered by phuclv on December 20, 2020

In zsh, you could define a math function like:

() {
  typeset -gA bsuffix
  local n=1 ni=1 s
  for s (k m g t p e) {
    (( n *= 1000 )); (( ni *= 1024 ))
    (( bsuffix[$s] = bsuffix[${s}ib] = bsuffix[${s}io] = ni ))
    (( bsuffix[${s}b] = bsuffix[${s}o] = n ))
  }
}
b() {
  set -o localoptions -o extendedglob
  local s=${(M)1%(#i)(${(j:|:k)~bsuffix})}
  (( ${1%$s} * ${bsuffix[$s:l]-1} ))
}

functions -Ms b

Then you'd be able to use b(1G), b(1mB) in any zsh arithmetic expression, like in (( .... )), $(( ... )), $array[...], etc, or in zcalc:

$ <<< $((b(86k) + b(320mb) + b(1.7gio)))
2145449164.8

$ autoload zcalc
$ zcalc
1> b(86k) + b(320mb) + b(1.7gio)
2.14545e+09
2> :sci 15
2145449164.8

$ echo $(( b(infeo) ))
Inf   ?

(note that we make no difference between b and B (or o / O), the match it case insensitive. It's not interpreted as bit vs byte).

Another approach could be to have the b() function take the whole expression as argument, and replace all the suffixes with * $bsuffix[<suffix>]

b() {
  set -o localoptions -o extendedglob
  local s=${(M)1%(#i)(${(j:|:k)~bsuffix})}
  (( ${1//(#bi)([0-9.][[:blank:]]#)(${(j:|:k)~bsuffix})/$match[1] * $bsuffix[$match[2]:l] } ))
}

And then:

$ echo $(( b(1m + 1Mb) ))
2048576

There's the problem of e/E (exa) though which puts a spanner in the works in that 1e-3GB would not be interpreted as 0.001 * 1000000000 but as 1 * 1152921504606846976 - 3 * 1000000000.

In any shell with support for floating point arithmetic (ksh93, zsh, yash), you could always define:

  K=1024  M=$((K * K))  G=$((M * K))  T=$((G * K))  P=$((T * K))  E=$((P * K))
KiB=$K  MiB=$M        GiB=$G        TiB=$T        PiB=$P        EiB=$E
 KB=1000 MB=$((KB*KB)) GB=$((MB*KB)) TB=$((GB*KB)) PB=$((TB*KB)) EB=$((PB*KB))

Or to golf it:

K=1024 EiB=$((E=K*(P=PiB=K*(T=TiB=K*(G=GiB=K*(M=MiB=K*K))))))
KB=1000 EB=$((EB=KB*(PB=KB*(TB=KB*(GB=KB*(MB=KB*KB))))))

And write $(( 1.1*GB + 5*K ))

to add the suffixes on output, you could use GNU numfmt:

$ human() numfmt --field=- --to=iec --suffix=iB
$ echo $(( b(1m + 1Mb) )) | human
2.0MiB

Answered by Stéphane Chazelas on December 20, 2020

There is Bcal.

$ bcal -m "(5kib+2mib)/2"
1051136 B
$ bcal -m "(5kb+2mb)/2"
1002500 B

The -m flag is for brief output. Removing it outputs verbosely with base 2 (KiB, MiB, GiB, TiB) and base 10 (kB, MB, GB, TB) results.

It does not understand 86k or 320m or 1.7g, after all those are not proper byte units. In that case, you could use Sed to add the b after each letter and then pipe it to bcal:

$ cat file
1.7g+320m+86k
$ sed 's/[gmk]/&b/g' file | bcal -m
bcal> 1.7gb+320mb+86kb
2020086000 B

You can also use it in interactive mode.

Answered by Quasímodo on December 20, 2020

Add your own answers!

Ask a Question

Get help from others!

© 2024 TransWikia.com. All rights reserved. Sites we Love: PCI Database, UKBizDB, Menu Kuliner, Sharing RPP