Security analysis of an encryption scheme based on nonpositional polynomial notations

The aim of the research was to conduct a cryptographic analysis of an encryption scheme developed on the basis of nonpositional polynomial notations to estimate the algorithm strength. Nonpositional polynomial notations (NPNs) are residue number systems (RNSs) based on irreducible polynomials over GF(2). To evaluate if the algorithms developed on the basis of NPNs are secure, mathematical models of cryptanalysis involving algebraic, linear and differential methods have been designed. The cryptanalysis is as follows. A system of nonlinear equations is obtained from a function transforming plaintext into ciphertext with a key. Next, a possibility of transition of the nonlinear system to a linear one is considered. The cryptanalysis was conducted for the cases with known: 1) ciphertext; 2) plaintext and the related ciphertext; 3) plaintext file format; and 4) ASCII-encoded plaintext.