The neural network designer must take into consideration many factors when selecting an appropriate network configuration. The performance of a given network configuration is influenced by many different factors such as: accuracy, training time, sensitivity, and the number of neurons used in the implementation. Using a cost function based on the four criteria mentioned previously, the various network paradigms can be evaluated relative to one another. If the mathematical models of the evaluation criteria as functions of the network configuration are known, then traditional techniques (such as the steepest descent method) could be used to determine the optimal network configuration. The difficulty in selecting an appropriate network configuration is due to the difficulty involved in determining the mathematical models of the evaluation criteria. This difficulty can be avoided by using fuzzy logic techniques to perform the network optimization as opposed to the traditional techniques. Fuzzy logic avoids the need of a detailed mathematical description of the relationship between the network performance and the network configuration, by using heuristic reasoning and linguistic variables. A comparison will be made between the fuzzy logic approach and the steepest descent method for the optimization of the cost function. The fuzzy optimization procedure could be applied to other systems where there is a priori information about their characteristics.
Hessian with Mini-Batches for Electrical Demand Prediction
