Since many years I had read many articles and chapters in different books about OSPF path selection and the same definition and the same logic is always covered as follow:
1-Intra-Area
2-Inter-Area
3-E1 or N1
4-E2 or N2
This traditional path selection is based on two criterias:
-ROUTE TYPE
-COST (between E1 and N1, E2 and N2)
Starting by investigating RFCs for OSPF, especially RFC 3101 and RFC 1587 for NSSA, RFC 1583 and RFC 2328 for OSPFv2. I discovered another step and level of path selection, which based on RFC, in other words when moving to E1/E2 versus N1/N2, another order of selection should be taken in consideration which RFC, more exactly, RFC 3101 and RFC 1587.
The changes are huge, the old RFC 1587 for NSSA which can be currently turned on using the compatible RFC1587 command, says that if a router receive a Type-7 LSA and Type-5 for the same destination the cost to the forwarding address or ASBR are equal, the Type-5 LSA is always preferred, the OE1/OE2 routes is preferred than ON1/ON2 routes. This path selection rule is only valid if RFC 1583 is enabled (this the default on Cisco Routers).
RFC 1587, section 3.5 Calculating Type-7 AS External Routes:
When a type-5 LSA and a type-7 LSA are found to have the
same type and an equal distance, the following priorities
apply (listed from highest to lowest) for breaking the tie.
a. Any type 5 LSA.
b. A type-7 LSA with the P-bit set and the forwarding
address non-zero.
c. Any other type-7 LSA.
The reason why the RFC 1587 said that RFC 1583 is valid, because when you enable RFC 1583, this means RFC 2328 is enabled, so when RFC 2328 is there , the behavior of RFC 1587 changes, and becames obsolete, while RFC 1583 is solely based on cost. RFC 2328 says the Intra-area route to ASBR through non-backbone is aways preferred than an intra-area through the backbone or inter-area to the ASBR.
Per RFC 2328, the following rules indicate which paths are preferred when multiple intra-AS paths are available to ASBRs or forwarding addresses:
1-Intra-area paths using nonbackbone areas are always the most preferred.
2-The other paths, intra-area backbone paths and inter-area paths, are of equal preference.
This means that the ON2 routes learned through an NSSA area is always preferred. While RFC 1587 tell us that the OE route is preferred than ON2 route, therefore both RFC 1587 and RFC 2328 cannot coexist, if they coexist RFC 2328 takes precedence over RFC 1587.
Now for the new RFC 3101, the order is inverted, in others words, if a router receive a Type-7 LSA and Type-5 for the same destination the cost to the forwarding address or ASBR are equal, the Type-7 LSA is always preferred, the ON1/ON2 routes is preferred than OE1/OE2 routes. For both RFC 1583 and RFC 2328, RFC 3101 can coexist with them.
RFC 3101, Section: 2.5 Calculating Type-7 AS External Routes says:
If the current LSA is functionally the same as an
installed LSA (i.e., same destination, cost and non-zero
forwarding address) then apply the following priorities in
deciding which LSA is preferred:
1. A Type-7 LSA with the P-bit set.
2. A Type-5 LSA.
3. The LSA with the higher router ID.
Demystify and Simplify 