That was my experience as well, and the moss choked them out. I tried small gravel as well, but I still had weeds in it, and landscape fabric wouldn't work for me because the bricks were 4 inches thick which is plenty of dirt for weeds to grow.