When looking at how much memory is free on a Solaris 10 server using ZFS, there is no accounting for ZFS caching and this needs to be done manually. Basically, the “Memory used” report on Solaris 10 platforms is useless when that platform is using ZFS.
For example, on my SunFire V490 with 64GB of physical memory, memory used reports that ~63 GB (+/- 1G:cool: is being used at any one time, but in reality, 25GB of this is ZFS caching. As far as ZFS goes, ZFS is allowed to use any available memory minus 1GB and will relinquish as necessary.
First, looking at mdb -k output:
# echo “::memstat” | mdb -k
Page Summary Pages MB %Tot
—-
Kernel 3289294 25697 40%
Anon 4199863 32811 51%
Exec and libs 99041 773 1%
Page cache 223967 1749 3%
Free (cachelist) 10795 84 0%
Free (freelist) 412599 3223 5%
Total 8235559 64340
Physical 8231550 64308
#
Kernel memory is more than 25GB and this is where ZFS does its caching.
So the quesion is, how much of this 25GB is actually ZFS caching? Well, the answer is virtually all of it.
# kstat zfs:0:arcstats:size
module: zfs instance: 0
name: arcstats class: misc
size 25796432032
#
This feature request is to add a metric under the Memory graphing which can report on ZFS caching statistics (kstat) so that administrators can accurately determine free memory on Solaris 10 platforms.