CS Lecture-Module #08:
Further Sorting Algorithms

merge-sort

outline of scheme:
recursively sort halves of given sequence,
then merge the two sorted halves into complete sorted sequence

advantages:

(see analysis below)

disadvantages:


I'll start presentation of details of algorithm with the following crucial function: 
merge(Comparable[] a, Comparable[] tmpArray,
        int leftPos, int rightPos, int rightEnd)
    {
    see below for content
    }
works on  a[leftPos .. rightEnd]
precondition: this segment's left half  a[leftPos .. (rightPos-1)]  is sorted,
and right half  a[rightPost .. rightEnd]  is sorted
e.g.: 
a[]:
. . . --+---+---+---+---+---+---+---+---+---+---+---+---+-- . . .
. . . . | . | . | A | E | K | R | C | H | U | Z | . | . | . . . .
. . . --+---+---+---+---+---+---+---+---+---+---+---+---+-- . . .
                  lp              rp          re
what  merge()  does is merge the two halves of this segment together
making the entire segment sorted
using  tmpArray[]  as temporary space

here's the code for  merge(), from page 239 of textbook by Weiss: 
/**
 * method that merges two sorted halves of a subarray.
 * @param |a| an array of |Comparable| items.
 * @param |tmpArray| an array to place the merged result.
 * @param |leftPos| the left-most index of the subarray.
 * @param |rightPos| the index of the start of the second half.
 * @param |rightEnd| the right-most index of the subarray.
 * @author Mark Allen Weiss.
 */
void merge( Comparable [ ] a, Comparable [ ] tmpArray,
           int leftPos, int rightPos, int rightEnd )
{
    int leftEnd = rightPos - 1;
    int tmpPos = leftPos;
    int numElements = rightEnd - leftPos + 1;

    // Main loop
    while( leftPos <= leftEnd && rightPos <= rightEnd )
        if( a[ leftPos ].compareTo( a[ rightPos ] ) <= 0 )
            tmpArray[ tmpPos++ ] = a[ leftPos++ ];
        else
            tmpArray[ tmpPos++ ] = a[ rightPos++ ];

    while( leftPos <= leftEnd )    // Copy rest of first half
        tmpArray[ tmpPos++ ] = a[ leftPos++ ];

    while( rightPos <= rightEnd )  // Copy rest of right half
        tmpArray[ tmpPos++ ] = a[ rightPos++ ];

    // Copy tmpArray back
    for( int i = 0; i < numElements; i++, rightEnd-- )
        a[ rightEnd ] = tmpArray[ rightEnd ];
}
here's an example of its processing: 
a[]:                                tmpArray[]:
+---+---+---+---+---+---+---+---+   +---+---+---+---+---+---+---+---+
|"A"|"E"|"K"|"R"|"C"|"H"|"U"|"Z"|   |   |   |   |   |   |   |   |   |
+---+---+---+---+---+---+---+---+   +---+---+---+---+---+---+---+---+
  0   1   2   3   4   5   6   7       0   1   2   3   4   5   6   7
  lp              rp          re     

merge(Comparable[] a, Comparable[] tmpArray,
            int leftPos, int rightPos, int rightEnd)
    {
    int leftEnd = rightPos - 1 ;
    int tmpPos = leftPos ;
    int numElements = rightEnd - leftPos + 1 ;

a[]:                                tmpArray[]:
+---+---+---+---+---+---+---+---+   +---+---+---+---+---+---+---+---+
|"A"|"E"|"K"|"R"|"C"|"H"|"U"|"Z"|   |   |   |   |   |   |   |   |   |
+---+---+---+---+---+---+---+---+   +---+---+---+---+---+---+---+---+
  0   1   2   3   4   5   6   7       0   1   2   3   4   5   6   7
  lp          le  rp          re      tp

    

    // Main loop
    while ( leftPos <= leftEnd && rightPos <= rightEnd   )
        if ( a[leftPos].compareTo(a[rightPos]) <= 0  )
            tmpArray[tmpPos++ ] = a[leftPos++ ] ;
        else
            tmpArray[tmpPos++] = a[rightPos++];

a[]:                                tmpArray[]:
+---+---+---+---+---+---+---+---+   +---+---+---+---+---+---+---+---+
|"A"|"E"|"K"|"R"|"C"|"H"|"U"|"Z"|   |"A"|   |   |   |   |   |   |   |
+---+---+---+---+---+---+---+---+   +---+---+---+---+---+---+---+---+
  0   1   2   3   4   5   6   7       0   1   2   3   4   5   6   7
      lp      le  rp          re          tp

    while ( leftPos <= leftEnd && rightPos <= rightEnd   )
        if ( a[leftPos].compareTo(a[rightPos]) <= 0   )
            tmpArray[tmpPos++] = a[leftPos++];
        else
            tmpArray[tmpPos++] = a[rightPos++] ;

a[]:                                tmpArray[]:
+---+---+---+---+---+---+---+---+   +---+---+---+---+---+---+---+---+
|"A"|"E"|"K"|"R"|"C"|"H"|"U"|"Z"|   |"A"|"C"|   |   |   |   |   |   |
+---+---+---+---+---+---+---+---+   +---+---+---+---+---+---+---+---+
  0   1   2   3   4   5   6   7       0   1   2   3   4   5   6   7
      lp      le      rp      re              tp

    while ( leftPos <= leftEnd && rightPos <= rightEnd   )
        if ( a[leftPos].compareTo(a[rightPos]) <= 0   )
            tmpArray[tmpPos++] = a[leftPos++] ;
        else
            tmpArray[tmpPos++] = a[rightPos++];

a[]:                                tmpArray[]:
+---+---+---+---+---+---+---+---+   +---+---+---+---+---+---+---+---+
|"A"|"E"|"K"|"R"|"C"|"H"|"U"|"Z"|   |"A"|"C"|"E"|   |   |   |   |   |
+---+---+---+---+---+---+---+---+   +---+---+---+---+---+---+---+---+
  0   1   2   3   4   5   6   7       0   1   2   3   4   5   6   7
          lp  le      rp      re                  tp

    while ( leftPos <= leftEnd && rightPos <= rightEnd   )
        if ( a[leftPos].compareTo(a[rightPos]) <= 0   )
            tmpArray[tmpPos++] = a[leftPos++];
        else
            tmpArray[tmpPos++] = a[rightPos++] ;

a[]:                                tmpArray[]:
+---+---+---+---+---+---+---+---+   +---+---+---+---+---+---+---+---+
|"A"|"E"|"K"|"R"|"C"|"H"|"U"|"Z"|   |"A"|"C"|"E"|"H"|   |   |   |   |
+---+---+---+---+---+---+---+---+   +---+---+---+---+---+---+---+---+
  0   1   2   3   4   5   6   7       0   1   2   3   4   5   6   7
          lp  le          rp  re                      tp

    while ( leftPos <= leftEnd && rightPos <= rightEnd   )

a[]:                                tmpArray[]:
+---+---+---+---+---+---+---+---+   +---+---+---+---+---+---+---+---+
|"A"|"E"|"K"|"R"|"C"|"H"|"U"|"Z"|   |"A"|"C"|"E"|"H"|"K"|   |   |   |
+---+---+---+---+---+---+---+---+   +---+---+---+---+---+---+---+---+
  0   1   2   3   4   5   6   7       0   1   2   3   4   5   6   7
              le          rp  re                          tp
              lp

    while ( leftPos <= leftEnd && rightPos <= rightEnd   )

a[]:                                tmpArray[]:
+---+---+---+---+---+---+---+---+   +---+---+---+---+---+---+---+---+
|"A"|"E"|"K"|"R"|"C"|"H"|"U"|"Z"|   |"A"|"C"|"E"|"H"|"K"|"R"|   |   |
+---+---+---+---+---+---+---+---+   +---+---+---+---+---+---+---+---+
  0   1   2   3   4   5   6   7       0   1   2   3   4   5   6   7
              le          rp  re                          tp
                  lp

    while ( leftPos <= leftEnd && rightPos <= rightEnd   )
        // so conclude executing that loop

    // next:

    while ( leftPos <= leftEnd   )       // Copy rest of first half
        tmpArray[tmpPos++] = a[leftPos++];

    while ( rightPos <= rightEnd  )       // Copy rest of right half
        tmpArray[tmpPos++] = a[rightPos++];

a[]:                                tmpArray[]:
+---+---+---+---+---+---+---+---+   +---+---+---+---+---+---+---+---+
|"A"|"E"|"K"|"R"|"C"|"H"|"U"|"Z"|   |"A"|"C"|"E"|"H"|"K"|"R"|"U"|   |
+---+---+---+---+---+---+---+---+   +---+---+---+---+---+---+---+---+
  0   1   2   3   4   5   6   7       0   1   2   3   4   5   6   7
              le              re                                  tp
                  lp          rp

    while ( rightPos <= rightEnd  )       // Copy rest of right half
        tmpArray[tmpPos++] = a[rightPos++];

a[]:                                tmpArray[]:
+---+---+---+---+---+---+---+---+   +---+---+---+---+---+---+---+---+
|"A"|"E"|"K"|"R"|"C"|"H"|"U"|"Z"|   |"A"|"C"|"E"|"H"|"K"|"R"|"U"|"Z"|
+---+---+---+---+---+---+---+---+   +---+---+---+---+---+---+---+---+
  0   1   2   3   4   5   6   7   8   0   1   2   3   4   5   6   7   8
              le              re                                      tp
                  lp              rp

    while ( rightPos <= rightEnd  )
        // so conclude executing that loop

    // Copy tmpArray[] back
    for ( int i = 0; i < numElements; i++, rightEnd-- )
        a[rightEnd] = tmpArray[rightEnd];

a[]:                                tmpArray[]:
+---+---+---+---+---+---+---+---+   +---+---+---+---+---+---+---+---+
|"A"|"C"|"E"|"H"|"K"|"R"|"U"|"Z"|   |"A"|"C"|"E"|"H"|"K"|"R"|"U"|"Z"|
+---+---+---+---+---+---+---+---+   +---+---+---+---+---+---+---+---+
  0   1   2   3   4   5   6   7       0   1   2   3   4   5   6   7
  re                                  re

    } // end |merge()|

here's the remaining code for  mergeSort(), from page 238 of textbook by Weiss: 
mergeSort(Comparable[] a) {
    Comparable[] tmpArray = new Comparable[a.length];

    mergeSort(a, tmpArray, 0, a.length - 1);
    }

mergeSort(Comparable[] a, Comparable[] tmpArray,
                int left, int right)
    {
    if ( left < right ) {
        int center = (left + right) / 2;
        mergeSort(a, tmpArray, left, center);
        mergeSort(a, tmpArray, center + 1, right);
        merge(a, tmpArray, left, center + 1, right);
        }
    }
example of sketch of processing
(not discussing all details, just doing overview here): 
level-0 invocation of mergeSort() gets initial data:
+---+---+---+---+---+---+---+---+
| K | E | R | A | H | Z | U | C |
+---+---+---+---+---+---+---+---+

    first level-1 recursive invocation will work on first half:
    +---+---+---+---+---+---+---+---+
    | K | E | R | A | . | . | . | . |
    +---+---+---+---+---+---+---+---+

        first level-2 recursive invocation here will work on first sub-half
        i.e. first quarter:
        +---+---+---+---+---+---+---+---+
        | K | E | . | . | . | . | . | . |
        +---+---+---+---+---+---+---+---+
        yielding the following:
        +---+---+---+---+---+---+---+---+
        | E | K | . | . | . | . | . | . |
        +---+---+---+---+---+---+---+---+

        second level-2 recursive invocation here will work on second sub-half
        i.e. second quarter:
        +---+---+---+---+---+---+---+---+
        | . | . | R | A | . | . | . | . |
        +---+---+---+---+---+---+---+---+
        yielding the following:
        +---+---+---+---+---+---+---+---+
        | . | . | A | R | . | . | . | . |
        +---+---+---+---+---+---+---+---+

    currently things appear as follows:
    +---+---+---+---+---+---+---+---+
    | E | K | A | R | . | . | . | . |
    +---+---+---+---+---+---+---+---+
    then merge() yields the following:
    +---+---+---+---+---+---+---+---+
    | A | E | K | R | . | . | . | . |
    +---+---+---+---+---+---+---+---+

    second level-1 recursive invocation will work on second half:
    +---+---+---+---+---+---+---+---+
    | . | . | . | . | H | Z | U | C |
    +---+---+---+---+---+---+---+---+

        first level-2 recursive invocation here will work
        on first sub-half here:
        +---+---+---+---+---+---+---+---+
        | . | . | . | . | H | Z | . | . |
        +---+---+---+---+---+---+---+---+
        (not much to do here)

        second level-2 recursive invocation here will work
        on second sub-half here:
        +---+---+---+---+---+---+---+---+
        | . | . | . | . | . | . | U | C |
        +---+---+---+---+---+---+---+---+
        yielding the following:
        +---+---+---+---+---+---+---+---+
        | . | . | . | . | . | . | C | U |
        +---+---+---+---+---+---+---+---+

    then the HZ.. and ..CU need to be merged, yielding:
    +---+---+---+---+---+---+---+---+
    | . | . | . | . | C | H | U | Z |
    +---+---+---+---+---+---+---+---+

then the AEKR.... and ....CHUZ need to be merged, yielding:
+---+---+---+---+---+---+---+---+
| A | C | E | H | K | R | U | Z |
+---+---+---+---+---+---+---+---+
time-analysis:
mergeSort()  does 2 recursive invocations of itself
each on half of the given array
plus 1 invocation of  merge()

first discuss the  merge():
merge()  has index-variables  leftPos,  rightPos,  tmpPos
which pass through arrays  a[],  tmpArray[]
leftPos  through left half of  a[],
rightPos  through right half of  a[]
    totaling one pass through  a[]  -- 
tmpPos  through all of  tmpArray[]  -- 
plus  i/rightEnd  copying from  tmpArray[]  back to  a[]  at end
    -- 
thus time-complexity of  merge()  is O() 
               mergeSort()
               merge(): N               total work at this level:  N
    +---+---+---+---+---+---+---+---+
    | K | E | R | A | H | Z | U | C |
    +---+---+---+---+---+---+---+---+
mergeSort()  makes 2 recursive invocations on halves with lengths N/2
each of the 2 invocations does a  merge()  which is O(N/2) totaling O(N): 
       mergeSort()     mergeSort()
       merge(): N/2    merge(): N/2     total: 
    +---+---+---+---+---+---+---+---+
    | K | E | R | A | H | Z | U | C |
    +---+---+---+---+---+---+---+---+
but also each of those 2 invocations makes another 2 sub-invocations each
so will need to add time for that work
well, each of these 4 invocations does a  merge()  which is O(N/4) totaling O(N): 
      mergeSort():     mergeSort():
      ms()    ms()     ms()    ms()
     m():N/4 m():N/4 m():N/4 m():N/4    total: 
    +---+---+---+---+---+---+---+---+
    | K | E | R | A | H | Z | U | C |
    +---+---+---+---+---+---+---+---+

    and so on
can similarly analyze all further levels of recursive invocations
could say many invocations reach each level at some point...
but if you add up all the work done at each level, get N
then total work should be O(N * num_levels)

what is num_levels?
recursion stops if  mergeSort()  invoked on num_elements 1 or 0
num_levels reflects number of times can divide N by 2 until reach 1 (or 0)
what is this value? 

thus time for  mergeSort()  is O()

quicksort

general idea/scheme:
split sequence of elements into small ones and large ones
-- as compared to some nicely chosen "pivot" value --
then recursively sort the resultant two subsequences

example with implementation pages 246-47 of textbook by Weiss: 
quickSort(Comparable[] a, int left, int right) {

    H  U  G  H  W  I  N  G  F  I  E  L  D  M  C  G  U  I  R  E
    0  1  2  3  4  5  6  7  8  9 10 11 12 13 14 15 16 17 18 19  
    l                                                        r

    // see discussion of CUTOFF below

    Comparable pivot = median3(a, left, right) {
        /**
         * Return median of left, center, and right.
         * Order these and hide the pivot.
         */

        int center = (left + right)/2;

    H  .  .  .  .  .  .  .  .  I  .  .  .  .  .  .  .  .  .  E
    0  1  2  3  4  5  6  7  8  9 10 11 12 13 14 15 16 17 18 19  
    l                          c                             r

        if ( a[center].compareTo(a[left]) < 0   )
            swapReferences(a, left, center);

    H  .  .  .  .  .  .  .  .  I  .  .  .  .  .  .  .  .  .  E
    l                          c                             r

        // if original material was  x...y...z,
        // then |a[left]| now contains min{x,y}

        if ( a[right].compareTo(a[left]) < 0   )
            swapReferences(a, left, right);

    E  .  .  .  .  .  .  .  .  I  .  .  .  .  .  .  .  .  .  H
    l                          c                             r

        // that compared z which was |a[right]| to
        // |a[left]| which was min{x,y}
        // and if z < min{x,y}, put z in |a[left]|
        // so now |a[left]| contains smallest of {x,y,z}
        // and |a[center]|, |a[right]| contain largest two of the values

        // the next chunk of code puts a[center],a[right] in order:
        if ( a[right].compareTo(a[center]) < 0  )
            swapReferences(a, center, right);

    E  .  .  .  .  .  .  .  .  H  .  .  .  .  .  .  .  .  .  I
    l                          c                             r


        // Place pivot at position  right - 1 :

    E  .  .  .  .  .  .  .  .  H  .  .  .  .  .  .  .  .  R  I
    l                          c                         r-1 r

        swapReferences(a, center, right - 1);

    E  .  .  .  .  .  .  .  .  R  .  .  .  .  .  .  .  .  H  I
    l                          c                         r-1 r

        return a[right - 1];
        }
    Comparable pivot = median3(a, left, right);

    // Begin partitioning
    int i = left, j = right - 1;

    l                                                    r-1 r
    E  U  G  H  W  I  N  G  F  R  E  L  D  M  C  G  U  I  H  I
    i                                                     j

    for ( ; ; ) {

        // shift |i| rightward while the element at |i|
        // is less than the pivot:
        while ( a[++i].compareTo(pivot) < 0 ) { }

    E  U  G  H  W  I  N  G  F  R  E  L  D  M  C  G  U  I  H  I
       |                                                  j  r
       |                                         
       first a[++i]                             
       i left here                             

        // shift |j| leftward while the element at |j|
        // is greater than the pivot:
        while ( a[--j].compareTo(pivot) > 0 ) { }

    E  U  G  H  W  I  N  G  F  R  E  L  D  M  C  G  U  I  H  I
       |                                         | ... |     r
       |                                         |     |
       i left here                               |     first a[--j]
                                                 j left here

        if ( i < j   )
            swapReferences(a, i, j);
        else
            break;

       i                                         j
    E  G  G  H  W  I  N  G  F  R  E  L  D  M  C  U  U  I  H  I

        }
    for ( ; ; ) {

        // shift |i| rightward while the element at |i|
        // is less than the pivot:
        while ( a[++i].compareTo(pivot) < 0 ) { }

        // shift |j| leftward while the element at |j|
        // is greater than the pivot:
        while ( a[--j].compareTo(pivot) > 0 ) { }

    E  G  G  H  W  I  N  G  F  R  E  L  D  M  C  U  U  I  H  I
             i                                j

        if ( i < j   )
            swapReferences(a, i, j);
        else
            break;

             i                                j
    E  G  G  C  W  I  N  G  F  R  E  L  D  M  H  U  U  I  H  I

        }
    for ( ; ; ) {
        while ( a[++i].compareTo(pivot) < 0 ) { }
        while ( a[--j].compareTo(pivot) > 0 ) { }

    E  G  G  C  W  I  N  G  F  R  E  L  D  M  H  U  U  I  H  I
                i                       j

        if ( i < j   )
            swapReferences(a, i, j);
        else
            break;

                i                       j
    E  G  G  C  D  I  N  G  F  R  E  L  W  M  H  U  U  I  H  I

        }
    for ( ; ; ) {
        while ( a[++i].compareTo(pivot) < 0 ) { }
        while ( a[--j].compareTo(pivot) > 0 ) { }

    E  G  G  C  D  I  N  G  F  R  E  L  W  M  H  U  U  I  H  I
                   i              j

        if ( i < j   )
            swapReferences(a, i, j);
        else
            break;

                   i              j
    E  G  G  C  D  E  N  G  F  R  I  L  W  M  H  U  U  I  H  I

        }
    for ( ; ; ) {
        while ( a[++i].compareTo(pivot) < 0 ) { }
        while ( a[--j].compareTo(pivot) > 0 ) { }

    E  G  G  C  D  E  N  G  F  R  I  L  W  M  H  U  U  I  H  I
                      i     j

        if ( i < j   )
            swapReferences(a, i, j);
        else
            break;

                      i     j
    E  G  G  C  D  E  F  G  N  R  I  L  W  M  H  U  U  I  H  I

        }
    for ( ; ; ) {
        while ( a[++i].compareTo(pivot) < 0 ) { }

    E  G  G  C  D  E  F  G  N  R  I  L  W  M  H  U  U  I  H  I
                            i
                            j

        while ( a[--j].compareTo(pivot) > 0 ) { }

    E  G  G  C  D  E  F  G  N  R  I  L  W  M  H  U  U  I  H  I
                            i
                         j

        if ( i < j  )
            swapReferences(a, i, j);
        else
            break;
        }

    swapReferences(a, i, right - 1);    // Restore pivot

                            i
                         j                               r-1 r
    E  G  G  C  D  E  F  G  H  R  I  L  W  M  H  U  U  I  N  I

    quicksort(a, left, i - 1) {         // Sort small elements

                            i
    l                   i-1 i
    E  G  G  C  D  E  F  G  .  .  .  .  .  .  .  .  .  .  .  .

        // yields:

    C  D  E  E  F  G  G  G  .  .  .  .  .  .  .  .  .  .  .  .

        }

    quicksort(a, i + 1, right) {                // Sort large elements

                            i i+1                            r
    .  .  .  .  .  .  .  .  .  R  I  L  W  M  H  U  U  I  N  I

        // yields:

    .  .  .  .  .  .  .  .  .  H  I  I  I  L  M  N  R  U  U  W

        }

    // showing all the elements at this point, things are as follows:

    C  D  E  E  F  G  G  G  H  H  I  I  I  L  M  N  R  U  U  W
issue of cutoff:
quicksort's time is generally O(N*lg(N)) -- see below
but if N is small, quicksort suffers from overhead
of 'messy' loops etc.:  for ( ; ; )     while ( a[++i] )    while ( a[--j] )     if ( i < j )
plus recursive invocations -- function-calls are expensive (see CS 251)
so instead do 'tight-looped' insertion-sort

quicksort's time is generally N*lg(N)
by analysis same as for merge-sort
partitioning is O(N) work like merging (merge()) was O(N) work

but need qualifier "generally" because while merge-sort always definitely splits material in half for recursive calls,
quicksort might not split so perfectly in half if pivot is near beginning or end of sequence
e.g. if pivot ended up being C above then: 
    H  U  G  H  W  I  N  G  F  I  E  L  D  M  C  G  U  I  R  E

    C  U  G  H  W  I  N  G  F  I  E  L  D  M  H  G  U  I  R  E
in this case, partitioning would be doing one pass of selection-sort 
    .  U  G  H  W  I  N  G  F  I  E  L  D  M  H  G  U  I  R  E
and then if pivot ended up being D next, then: 
    .  D  G  H  W  I  N  G  F  I  E  L  U  M  H  G  U  I  R  E
in this case, partitioning doing another pass of selection-sort
and so on: 
    .  .  G  H  W  I  N  G  F  I  E  L  U  M  H  G  U  I  R  E
    .  .  E  E  W  I  N  G  F  I  H  L  U  M  H  G  U  I  R  G
    .  .  .  .  W  I  N  G  F  I  H  L  U  M  H  G  U  I  R  G
and so on
(or similarly if pivot were repeatedly a high element)
yielding time O(N²)
but in typical (human) applications for quicksort, pivot ends up being good

(Copyright © 2009 by Hugh McGuire   — for thoughts about this, see   http://www.cis.gvsu.edu/~mcguire/teaching/copyright_thoughts.html .)