In this study, we propose a computational diagnosis system for detecting the colorectal cancer from histopathological slices. The computational analysis was usually performed on patch level where only a small part of the slice is covered. However, slice-based classification is more realistic for histopathological diagnosis. The developed method combines both textural and structural features from patch images and proposes a two level classification scheme. In the first level, the patches in slices are classified into possible classes (adenomatous, inflamed, cancer and normal) and the distribution of the patches into these classes is considered as the information representing the slices. Then the slices are classified using a logistic linear classifier. In patch level, we obtain the correct classification accuracies of 94.36% and 96.34% for the cancer and normal classes, respectively. However, in slice level, the accuracies of the 79.17% and 92.68% are achieved for cancer and normal classes, respectively.