Hi,
To be honest with you, not at all. As you might already aware, CleanMart and MaterialMart are both just VM Themes. You also have a Joomla Template and bunch of modules and plugins. I suppose, the slowness is caused by one of those extensions. How is the speed when you temporarily disable MaterialMart from your website? (i bet would be the same)
MaterialMArt and CleanMart,both of them work based on "view" manipulation, never make a database call, never try to make something to slow your server down. They just display the elements that Virtuemart Core provides on the page.
Regards