In a large pot, heat the olive oil over medium heat. Add the onion and garlic, and sauté until the onion is translucent.
Add the cauliflower florets and vegetable broth to the pot. Bring to a boil, then reduce the heat and simmer until the cauliflower is tender, about 20 minutes.
Use an immersion blender to puree the soup until smooth. Stir in the milk or cream, and season with salt and pepper to taste.
Serve hot, garnished with your choice of toppings such as croutons, cheese, or parsley.