adding and remove classes on click events

I am working on a click event, which intially is pretty standard.

on.click: Remove the active class from all list items, then add active to the clicked list item.

ie

var li = $('.child-item');
li.click(function(){
  li.removeClass('active');
  li.addClass('active');
});  

Works as expected. However, I also need a condition that if the active class is clicked again, it will remove the active class from the clicked item.

I tried this:

$('.child-item').toggle(function(e){

   $('.child-item').removeClass('active');
   var filterValue = $(this).attr('data-filter');
   $(this).addClass('active');

}, function() {
   $('.child-item').removeClass('active');
   $(this).removeClass('active');           
});    

but often the toggle is in the off state, so it requires a double click to get the toggle back on track. This seems like it would be a pretty straightforward solution, but I cannot seem to get it to work.

Added a jsFiddle here with another option. This is fine for when the same element is clicked, but it doesn't clear all active classes on each click.