Open your theme functions.php and add the following code:

function xsimu_breadcrumb() {
        echo ' <ul id="breadcrumbs">';
    if (!is_home()) {
        echo '<li> <a href="';
        echo get_option('home');
        echo '">';
        echo 'Home';
        echo "</a><span class='separator'>»</span></li>";
        if (is_category() || is_single()) {
            echo ' <li>';
            the_category('<span class="separator">»</span></li><li> ');
            if (is_single()) {
                echo "</li><li><span class='separator'>»</span>";
                the_title();
                echo '</li>';
            }
        } elseif (is_page()) {
            echo '<li>';
            echo the_title();
            echo '</li>';
        }
    }
    elseif (is_tag()) {single_tag_title();}
    elseif (is_day()) {echo"<li>Archive for "; the_time('F jS, Y'); echo'</li>';}
    elseif (is_month()) {echo"<li>Archive for "; the_time('F, Y'); echo'</li>';}
    elseif (is_year()) {echo"<li>Archive for "; the_time('Y'); echo'</li>';}
    elseif (is_author()) {echo"<li>Author Archive"; echo'</li>';}
    elseif (isset($_GET['paged']) && !empty($_GET['paged'])) {echo "<li>Blog Archives"; echo'</li>';}
    elseif (is_search()) {echo"<li>Search Results"; echo'</li>';}
    echo '</ul>';
}

Add the following function in one of your templates. Most times you will use the header.php

<?php xsimu_breadcrumb(); ?>

Congratulations. You are done;)