Oil shale reservoirs are characterized by significant heterogeneity in mineral components and pronounced anisotropy in micromechanical properties—both influencing resource recovery. We couple fine-scale nanoindentation and mineral analyzer (Tescan Integrated Mineral Analyzer (TIMA)) profiling of the mechanical properties and components of oil shale samples from the Ordos Basin, China. We use an updated clustering method, including a more precise way to delineate mineral boundaries, to precisely categorize the numerous nanoindentation test data into mineral composition groups. The lowest-to-highest ranking of Young's modulus and fracture toughness values in our samples is in the order clay, quartz, feldspar, dolomite, and then pyrite. Anisotropic characteristics of each phase were determined at various scales, with values of Young's modulus and fracture toughness are higher on surfaces parallel to the bedding plane than on those perpendicular to it. The clay-rich dark phase exhibits lower Young's modulus, making its pore structures more prone to collapse during gas depletion. Conversely, the fracture toughness of the bright phase is higher than that of the dark phase, causing the hydraulic fracturing to more easily penetrate through the dark phase and stop at the bright phase boundary. These divergences in mechanical properties are caused by the microstructure of the oil shale during sedimentation: the discrete distribution of hard minerals in the bright phase constrains deformation, while the lamellar clay layers in the dark phase provide less restriction. Upgraded mesoscopic mechanical parameters obtained from the modified Mori-Tanaka method, incorporating a shape factor, return results close to reality. Young's modulus and fracture toughness are lower at the mesoscale than at the microscale, indicating greater rigidity and toughness in fine structures. This study provides important insights into the cross-scale deformation and fracture behavior of shale, highlighting its impact on reservoir deformation, fracture propagation, and oil recovery efficiency.