Put a large pot on the stove with water and bring to a boil. While waiting for the water to boil, place the vinegar, dijon, shallot, honey, salt, pepper and horseradish into a small bowl and whisk well to combine. Drizzle in olive oil and whisk constantly until thoroughly combined (or you can blend it all in a blender!)
Put the beets in the boiling water and cook for about 20 minutes, or until they are easily pierced with a knife. Drain the beets and peel once they have slightly cooled - the skins should come off pretty easily. Cut the beets into small wedges and set aside.
Peel and pit the avocados. Slice them into thick pieces and lay on a serving plate. Squeeze the lemon over top of the avocado slices - this will help keep from browning. Place the beets on top of the avocado.
Drizzle some of the dressing on top, to taste. Sprinkle with salt and pepper. Scatter the capers and chives on top.