And? Memory was never invented to do nothing.
Besides, it could well be pre-allocated memory in order to speed up things, as releasing and allocating costs cycles.
Had one of the 20 second hangs about an hour ago and was able to get a screenshot of the task manager window, it looks like the GPU fell asleep but the video usage was unchanged during the hang.
Gaming at 1080p, 60 Hz most settings high to ultra although I have seen over 90+% memory usage on occasion.