A Novel Mutual Authentication Protocol for H(e)NB and Mobile Devices Using S8 S-box

Abstract

In wireless communication, Femtocells are used to enhance the service coverage inside the office or home, particularly where access is narrow or engaged. Femtocells enhance the service quality that is attractive for the client and the mobile operator. Currently, security of core network from the unauthorized user in public environment is a big issue and there is no integrated solution to verify a rouge femtocell or unauthorized user equipment (UE). Forged equipment can launch any attack or hack the core network to act actively or passively. A femtocell is a feasible option to enhance the signal strength. Authentication of the legal user and legitimate femtocell is a serious issue. Many protocols have been proposed for mutual authentication but each has its own benefits and drawbacks. Hence, there is a need for a mutual authentication protocol, so communication equipments can verify each other during connection establishment and make a secure tunnel for communication. In this thesis we proposed a solution which provides mutual authentication between the User Equipment (UE) and femtocell that will make the system more secure. The proposed algorithm is interoperable and use S8 S-box for AES to reduce the authentication time. S-box transformation is an important part of the AES algorithm as it is the only nonlinear component which introduces confusion in the algorithm. S8 S-box can be obtained by applying symmetric group S8 on AES S-box, and 4032040320 new keys can be constructed. Every permutation of the elements of the S-box results in a new S-box which makes the communication more secure. Moreover, novel scheme is also compatible with the current network architecture and resist various network attacks e.g., Sybil attack, MITM, DOS, Eavesdropping & Injecting attack and Packet Sniffing.