I joined a friends game and saw a lot of leaves in the middle of his base.
My first instinct was to clear the area by gathering them. This did not work. Pressing E did nothing. When i talked to the host, he said to me "there are no leaves on my screen". µ
These leaves were removed before the first time i joined, any leaves removed after i joined i could see being removed or i could remove them myself.