Uludağ University Journal of The Faculty of Engineering, Vol. 26, No. 1, 2021 RESEARCH DOI: 10.17482/uumfd.797087 NOISE CANCELLATION WITH LMS VARIANTS Fahri VATANSEVER* Received: 18.09.2020; revised: 25.01.2021; accepted: 02.02.2021 Abstract: The noise is one of the important factors that affect negatively the operation of circuits and systems. There are different methods and techniques for cancelling/reducing/de-noising these unwanted signals that mix with the original signal. In this study, an application/simulator that performs noise cancellation operations with least mean-squares algorithms was designed. The application, which has powerful visual support and a user-friendly interactive interface, can make single or comparative noise cancellation operations and performance analyzes with different algorithms on noisy signals. The effects of the algorithm parameters can be observed, information about the related algorithms can be obtained via the developed application that can also be used for educational purpose. Keywords: Noise cancellation, LMS, adaptive filter, simulator. LMS Varyantlarıyla Gürültü Arındırma Öz: Devre ve sistemlerin işleyişlerini olumsuz yönde etkileyen önemli faktörlerden birisi de gürültülerdir. Orijinal işarete karışan bu istenmeyen işaretlerin arındırılması/azaltılması/temizlenmesi için farklı yöntem ve teknikler mevcuttur. Gerçekleştirilen çalışmada, en küçük karesel ortalama algoritmalarıyla gürültü arındırma işlemlerini gerçekleştiren bir uygulama/simülatör tasarlanmıştır. Güçlü görsel desteğe ve kullanıcı dostu etkileşimli arayüze sahip, eğitim amaçlı da kullanılabilen uygulama ile gürültülü işaretler üzerinde, farklı algoritmalarla tekli veya karşılaştırmaları gürültü arındırma işlemleri ve performans analizleri yapılabilmekte, algoritma parametrelerinin etkileri gözlemlenebilmekte ve ilgili algoritmalar hakkında bilgiler edinilebilmektedir. Anahtar Kelimeler: Gürültü arındırma, LMS, adaptif filtre, simülatör. 1. INTRODUCTION The main purpose of the noise cancellation/reduction process, which is used in wide areas such as communication, biomedical, control, signal/image processing, is to reduce or completely eliminate the undesired signal (noise) in the desired signal. The fundamental process used in de- noising in digital signal processing is filtering. For this purpose, finite impulse response (FIR) and infinite impulse response (IIR) digital filters are used. However, problems are encountered in de-noising process if filters which have fixed coefficients are used, and the performance rate decreases. Adaptive filters have been used frequently in recent years to eliminate these problems. The adaptive filters are filters with self-adjusting characteristics. These filters, which can be designed as analog, digital or hybrid (mixed), have advantages and disadvantages compared to each other. For example, while analog filters consume low power and respond quickly, almost all *Bursa Uludağ University, Faculty of Engineering, Electrical-Electronics Eng. Dept. 16059 Bursa/Turkey Correspondence author: Fahri Vatansever (fahriv@uludag.edu.tr) 153 Vatansever F.:Noise Cancellation with LMS Variants of them are implemented as digital (FIR or IIR) due to the complexity of the optimization algorithms. As a result of the rapid development in the computer world, computer-aided technologies (CAx) concepts (such as computer-aided engineering (CAE), computer-aided design (CAD), computer-aided manufacturing (CAM), computer-aided software engineering (CASE), computer- aided instruction (CAI), computer-aided learning (CAL) etc.) have currently gained significant place. Simulators, applications, web pages, etc. are developed in many areas using computer software (Maddala, 2010; Kaçar and Çankaya, 2010; Kaçar and Çankaya, 2012; Yalcin and Vatansever, 2016; Hatun and Vatansever, 2016; Vatansever and Yalcin, 2017; Kaçar et al., 2017; Vatansever and Hatun, 2019). In this study, an application/simulator that performs noise cancellation operations with different variants of the least mean-squares (LMS) algorithm is designed. The application, which is designed using MATLAB App Designer (MathWorks, 2018), can be also used for educational purposes, can perform single or comparative noise cancellation operations, observe the effects of algorithm parameters, and examine the information about algorithms. This paper is organized as follows: In Section 2, adaptive filters and theirs application areas are summarized. In Section 3 , LMS algorithm and its variants are briefly explained. In Section 4, noise cancellation process with LMS is presented. In Section 5, designed application/simulator is explained and sample applications are given. Finally, Section 6 contains conclusions. 2. ADAPTIVE FILTER The commonly used adaptive filter structure is given in Fig. 1. The error signal here is used to create the objective/performance function that the related adaptive algorithm will handle. Minimizing this function ensures filter output matches the desired signal. The mean square of the error signal is the popularly used cost function. Table 1 shows the basic application classes of adaptive filtering (Widrow and Stearns, 1985; Morales, 2011; Haykin, 2014; Diniz, 2020). 𝒅[𝒌] Adaptive 𝒚[𝒌] 𝒌: Iteration number 𝒙[𝒌] + 𝒙[𝒌]: Input signal filter − 𝒅[𝒌]: Desired signal 𝒆[𝒌] 𝒚[𝒌]: Filter output signal 𝒆[𝒌] = 𝑑[𝑘] − 𝑦[𝑘]: Error signal Adaptive algorithm Figure 1: The commonly used adaptive filter structure As can be seen from Table 1, adaptive filters have wide usage areas: audio and video signal processing, noise and echo cancellation, channel estimation, channel equalization, modeling, inverse modeling, system identification, linear prediction, signal enhancement, active noise control, adaptive notch filter, jammer suppression, speech coding, adaptive line enhancement (Morales, 2011; Farhang-Boroujeny, 2013). Many studies have been carried out with different methods and techniques in the field of noise cancellation (Widrow et al., 1975; Thakor and Zhu, 1991; Gorriz et al., 2009; Sankar et al., 2010; Romero et al., 2012; Guda et al., 2014; Bismor et al., 2016; Dixit and Nagaria, 2017; Lee et al., 2017; Qureshi et al., 2017; Maurya, 2018; Taha et al., 2018; Goel and Chandra, 2019; Maurya et al., 2019). 154 Uludağ University Journal of The Faculty of Engineering, Vol. 26, No. 1, 2021 Table 1. The basic application classes of adaptive filtering Identification (modeling) Inverse modeling System Plan System t input output System Adaptive System input Plant filter output + Adapti ve filter + Delay Interference canceling Prediction Signal Adaptive Output-1 Reference Adapti ve Output Delay + Signal f ilter + signal filter Output-2 3. LEAST MEAN-SQUARES ALGORITHM LMS, one of the most used algorithms in adaptive filter applications, was developed in 1960 by Prof. Bernard Widrow and his Ph.D. student Ted Hoff. The main properties of this algorithm, of which many variants have been developed, can be listed as simplicity, ease of implementation, low computational complexity/cost, robustness etc. (Farhang-Boroujeny, 2013; Poularikas, 2015; Diniz, 2020). Input Unit delay signal 𝒙 𝒙 𝒙 𝒙 𝒙𝑴−𝟏 𝒙[𝒌] 𝟎 𝒛−𝟏 𝟏 𝟐 𝑴−𝟐 𝒛−𝟏 𝒛−𝟏 Desired signal 𝒅[𝒌] 𝒘𝟎 Adaptive 𝒘 𝒘𝟏 𝑴−𝟏 weights Output signal + 𝒚[𝒌] + − + Adaptation algorithm 𝑵𝒆𝒘 𝒘𝒆𝒊𝒈𝒉𝒕𝒔 𝑶𝒍𝒅 𝒘𝒆𝒊𝒈𝒉𝒕𝒔 𝑨𝒅𝒂𝒑𝒕𝒂𝒕𝒊𝒐𝒏 = + 𝒘[𝒌 + 𝟏] 𝒘[𝒌] 𝒈𝒂𝒊𝒏 Error signal 𝒆[𝒌] = 𝒅[𝒌] − 𝒚[𝒌] Figure 2: Adaptive linear combiner (ALC) in transversal form The adaptive linear combiner (adaptive FIR filter, adaptive transversal filter, ADALINE) as shown in Fig. 2. 155 Vatansever F.:Noise Cancellation with LMS Variants 𝑥0 𝑥[𝑘] 𝑤0 𝑥1 𝑥[𝑘 − 1] 𝑤1 𝑥 = [ ⋮ ] = [ ] , 𝑤 = [⋮ ⋮ ] (1) 𝑥𝑀−1 𝑥[𝑘 −𝑀 + 1] 𝑤𝑀−1 The output of the filter is written by Eq. (2), with 𝑥 filter input and 𝑤 tap-weight (coefficients) vector. 𝑀−1 𝑦[𝑘] = ∑ 𝑤𝑖[𝑘]𝑥[𝑘 − 𝑖] = 𝑤 𝑇𝑥 = 𝑥𝑇𝑤 (2) 𝑖=0 The error signal, its square and the mean square (squared) error (expectation of the error power or energy) are defined by Eq. (3-5). 𝑒[𝑘] = 𝑑[𝑘] − 𝑦[𝑘] = 𝑑[𝑘] − 𝑤𝑇𝑥 = 𝑑[𝑘] − 𝑥𝑇𝑤 (3) 𝑒2[𝑘] = (𝑑[𝑘] − 𝑥𝑇𝑤)2 = 𝑑2[𝑘] − 2𝑑[𝑘]𝑥𝑇𝑤 + (⏟𝑥 𝑇 𝑤 ) 2 (4) 𝑤𝑇𝑥𝑥𝑇𝑤 𝐽 = 𝐸{𝑒2[𝑘]} = 𝐸{𝑑2[𝑘]} − 2⏟𝐸{ 𝑑 [ 𝑘 ]𝑥 𝑇 }𝑤 +𝑤𝑇⏟𝐸 {𝑥 𝑥 𝑇 }𝑤 (5) 𝑟𝑇 𝑅𝑑𝑥 𝑥 Since the correlation matrix 𝑅𝑥 is a symmetric and positive definite matrix then Eq. (5) is a quadratic function of 𝑤. This function has only one minimum solution known as the optimum Wiener solution (Wiener weight vector). This solution is obtained by equating the derivative of the Eq. (5) to zero. Thus, the gradient of the mean square error function in Eq. (6) is obtained and its solution is given in Eq. (7). ∂𝐸{𝑒2[𝑘]} ∂(𝐸{𝑑2[𝑘]}−2𝑟𝑇 𝑤+𝑤𝑇𝑅 𝑤) ∇𝐸{𝑒2[𝑘]}= = 𝑑𝑥 𝑥 = −2𝑟𝑇 ∂𝑤 ∂𝑤 𝑑𝑥 + 2𝑅𝑥𝑤 = 0 (6) 𝑤0 = 𝑅 −1 𝑥 𝑟𝑑𝑥 (7) Generally, Eq. (7) (Wiener-Hopf equations) solution is not applicable and alternative solutions should be used as in Table 2 (Widrow and Stearns, 1985; Zaknich, 2005; Sayed, 2008; Farhang- Boroujeny, 2013; Haykin, 2014; Poularikas, 2015). Table 2. The some iterative algorithms Method/algorithm Iteration equation Solution iteration 𝑤(𝑘 + 1) = 𝑤(𝑘) + 𝜇𝑅−1𝑥 {𝑟𝑑𝑥 − 𝑅𝑥𝑤(𝑘)} Newton's method in 𝜕𝐽(𝑤(𝑘))⁄𝜕𝑤(𝑘) 𝑤(𝑘 + 1) = 𝑤(𝑘) − gradient search 𝜕2𝐽(𝑤(𝑘))⁄𝜕𝑤2(𝑘) 𝑤(𝑘 + 1) = 𝑤(𝑘) + 𝜇(𝑘){𝜖(𝑘)𝐼 + 𝑅𝑥} −1{𝑟𝑑𝑥 − 𝑅𝑥𝑤(𝑘)} 𝑤(𝑘 + 1) = 𝑤(𝑘) + 𝜇{𝑟𝑑𝑥 − 𝑅𝑥𝑤(𝑘)} Steepest-descent 𝑤(𝑘 + 1) = 𝑤(𝑘) − 𝜇∇𝐽(𝑤(𝑘)) algorithm 𝑤(𝑘 + 1) = 𝑤(𝑘) + 𝜇(𝑘){𝑟𝑑𝑥 − 𝑅𝑥𝑤(𝑘)} 156 Uludağ University Journal of The Faculty of Engineering, Vol. 26, No. 1, 2021 𝜕 𝜕 𝑇 By using steepest-descent algorithm, where ∇= [ ⋯ ] is the gradient operator, the 𝜕𝑤0 𝜕𝑤𝑀−1 following LMS recursion obtained. 𝑤[𝑘 + 1] = 𝑤[𝑘] − 𝜇 ∇⏟𝐽 (𝑤 [ 𝑘 ]) (8) ∇𝐸{𝑒2[𝑘]} 𝜕𝑒2[𝑘] 𝜕𝑒[𝑘] 𝜕(𝑑[𝑘]−𝑤𝑇𝑥) ∇𝐸{𝑒2[𝑘]}= = 2𝑒[𝑘] = 2𝑒[𝑘] = −2𝑒[𝑘]𝑥 (9) 𝜕𝑤 𝜕𝑤 𝜕𝑤 𝑤[𝑘 + 1] = 𝑤[𝑘] + 2𝜇𝑒[𝑘]𝑥[𝑘] (10) Replacing the iteration index 𝑘 by the time index 𝑛 in Eq. (10) is rewritten as Eq. (11). 𝑤(𝑛 + 1) = 𝑤(𝑛) + 2⏟𝜇 𝑒(𝑛)𝑥(𝑛) (11) 𝜇 In Table 3, some variants of LMS algorithm, which also used in this study are given (Sayed, 2008; Farhang-Boroujeny, 2013; Poularikas, 2015; Diniz, 2020). Table 3. The some variants of LMS algorithm Algorithm Definition Parameters 𝑤(𝑛) = [𝑤0 … 𝑤 𝑇𝑀−1] : Filter taps Least Mean- (coefficients) at time 𝑛 Square 𝑥(𝑛) = [𝑥(𝑛) … 𝑥(𝑛 −𝑀 + 1)]𝑇: Input (𝐿𝑀𝑆) data/signal to the filter 𝑤(𝑛 + 1) = 𝑤(𝑛) + 2𝜇 𝑒(𝑛)𝑥(𝑛) ⏟ 𝑦(𝑛) = 𝑤𝑇(𝑛)𝑥(𝑥) : Filter output 𝜇 (LMS with 𝑒(𝑛) = 𝑑(𝑛) − 𝑦(𝑥) : Error constant step 𝑑(𝑛) : Desired data/signal size) 𝜇 : Step-size factor/parameter 𝑀 : Order of the filter LMS with time-varying 𝑤(𝑛 + 1) = 𝑤(𝑛) + 𝜇(𝑛)𝑒(𝑛)𝑥(𝑛) step size Normalized 𝜇 LMS 𝑤(𝑛 + 1) = 𝑤(𝑛) + 𝑒(𝑛)𝑥(𝑛) ‖ ‖2 ( )𝑇 ‖ ‖2 𝑥(𝑛) = 𝑥 𝑛 𝑥(𝑛) 𝑥(𝑛) (𝑁𝐿𝑀𝑆) Modified version of the 𝜇 normalized 𝑤(𝑛 + 1) = 𝑤(𝑛) + 𝑒(𝑛)𝑥(𝑛) 2 𝜀 : Small positive number (constant) 𝜀 + ‖𝑥(𝑛)‖ LMS (𝜀 − 𝑁𝐿𝑀𝑆) Power 𝜇 𝑝(𝑛 + 1) = 𝑎𝑝(𝑛) + (1 − 𝑎)|𝑥(𝑛)| 2 normalized 𝑤(𝑛 + 1) = 𝑤(𝑛) + 𝑒(𝑛)𝑥(𝑛) 𝑝(0) = 0 LMS 𝜀 + 𝑝(𝑛 + 1) 0 ≤ 𝑎 < 1 (𝑃𝑁𝐿𝑀𝑆) Self- correcting The arrangement the standard LMS filter in a series form LMS filter (𝑆𝐶𝑁𝐿𝑀𝑆𝐹) 1 , 𝑓 > 0 Sign-error 𝑤(𝑛 + 1) = 𝑤(𝑛) + 𝜇{𝑠𝑖𝑔𝑛[𝑒(𝑛)]}𝑥(𝑛) 𝑠𝑖𝑔𝑛(𝑓) = { 0 , 𝑓 = 0 LMS −1 , 𝑓 < 0 Sign-error {𝑠𝑖𝑔𝑛[𝑒(𝑛)]}𝑥(𝑛) 𝑤(𝑛 + 1) = 𝑤(𝑛) + 𝜇 NLMS 𝜀 + ‖𝑥(𝑛)‖2 Dual-sign 𝜖 𝑠𝑖𝑔𝑛(𝑓) , |𝑓| > 𝛼 𝑤(𝑛 + 1) = 𝑤(𝑛) + 𝜇{𝑑𝑠[𝑒(𝑛)]}𝑥(𝑛) 𝑑𝑠(𝑓) = { algorithm 𝑠𝑖𝑔𝑛(𝑓) , |𝑓| ≤ 𝛼 157 Vatansever F.:Noise Cancellation with LMS Variants Sign- regressor LMS (Sign- 𝑤(𝑛 + 1) = 𝑤(𝑛) + 𝜇𝑒(𝑛){𝑠𝑖𝑔𝑛[𝑥(𝑛)]} data algorithm) Self- correcting The arrangement the standard sign-regressor LMS filter in a series sign- form regressor LMS Normalized sign- 𝑒(𝑛){𝑠𝑖𝑔𝑛[𝑥(𝑛)]} 𝑤(𝑛 + 1) = 𝑤(𝑛) + 𝜇 regressor 𝜀 + ‖𝑥(𝑛)‖2 LMS Sign-sign 𝑤(𝑛 + 1) = 𝑤(𝑛) + 𝜇{𝑠𝑖𝑔𝑛[𝑒(𝑛)]}{𝑠𝑖𝑔𝑛[𝑥(𝑛)]} LMS Normalized {𝑠𝑖𝑔𝑛[𝑒(𝑛)]}{𝑠𝑖𝑔𝑛[𝑥(𝑛)]} sign-sign 𝑤(𝑛 + 1) = 𝑤(𝑛) + 𝜇 ‖ LMS 𝜀 + 𝑥(𝑛)‖ 2 𝑝𝑤(𝑓) Power-of-two 𝑠𝑖𝑔𝑛(𝑓) , |𝑓| ≥ 1 error 𝑤(𝑛 + 1) = 𝑤(𝑛) + 𝜇{𝑝𝑤[𝑒(𝑛)]}𝑥(𝑛) = {2𝑓𝑙𝑜𝑜𝑟(log2|𝑓|)𝑠𝑖𝑔𝑛(𝑓) , 21−𝑓𝑑 ≤ |𝑓| < 1 algorithm 𝜏 𝑠𝑖𝑔𝑛(𝑓) , |𝑓| < 21−𝑓𝑑 Variable 𝑖 = 0,1,… ,𝑀 − 1 step-size 𝑤𝑖(𝑛 + 1) = 𝑤𝑖(𝑛) + 𝜇𝑖(𝑛)𝑒(𝑛)𝑥(𝑛 − 𝑖) 𝜇0(𝑛) 0 LMS 𝒘(𝑛 + 1) = 𝒘(𝑛) + 𝝁(𝑛)𝑒(𝑛)𝒙(𝑛) 𝜇(𝑛) = [ ⋱ ] (𝑉𝑆𝐿𝑀𝑆) 0 𝜇𝑀−1(𝑛) Leaky LMS 𝑤(𝑛 + 1) = (1 − 𝜇𝛾)𝑤(𝑛) + 𝜇𝑒(𝑛)𝑥(𝑛) 𝛾 : Leakage coefficient (0 < 𝛾 ≪ 1) Modified 𝑤(𝑛 + 1) = 𝑤(𝑛) + 𝜇𝐾𝑥(𝑛)𝑒2𝐾−1(𝑛) 𝐾 : Positive integer LMS Momentum 𝑤(𝑛 + 1) = 𝑤(𝑛) + (1 − 𝑔){𝑤(𝑛) − 𝑤(𝑛 − 1)} 0 < 𝑔 < 1 LMS + 𝑎𝑔𝜇𝑒(𝑛)𝑥(𝑛) 1 < 𝑎 Error 𝐿−1 normalized 𝜇𝑤(𝑛 + 1) = 𝑤(𝑛) + 𝑒(𝑛)𝑥(𝑛) ‖𝑒 (𝑛)‖2 =∑|𝑒(𝑛 − 𝑖)|2 step-size 1 + 𝜇‖𝑒 2 𝐿𝐿(𝑛)‖ LMS (𝐸𝑁𝑆𝑆) 𝑖=0 Robust 2 variable step- 𝜇‖𝑒𝐿(𝑛)‖ 𝑤(𝑛 + 1) = 𝑤(𝑛) + 𝑒(𝑛)𝑥(𝑛) size LMS 𝛼‖𝑒(𝑛)‖2 + (1 − 𝛼)‖𝑥(𝑛)‖2 (𝑅𝑉𝑆𝑆) Error-data normalized 𝑤(𝑛 + 1) step-size 𝜇= 𝑤(𝑛) + 𝑒(𝑛)𝑥(𝑛) LMS 𝛼‖𝑒 2𝐿(𝑛)‖ + (1 − 𝛼)‖𝑥(𝑛)‖ 2 (𝐸𝐷𝑁𝑆𝑆) Linearly 𝑎 − 𝑐𝑇𝑤′(𝑛) 𝑤′(𝑛) = 𝑤(𝑛) + 𝜇𝑒(𝑛)𝑥(𝑛) constrained 𝑤(𝑛 + 1) = 𝑤′(𝑛) + 𝑐 𝑇 𝑎 : Constant , 𝑐 : Constant vector LMS 𝑐 𝑐 Least mean 2 𝑤(𝐿𝑀𝐹 𝑛 + 1 ) = 𝑤(𝑛) + 𝜇𝑥(𝑛)𝑒(𝑛)(𝑒(𝑛) fourth ( ) ) Least mean 2 mixed norm 𝑤(𝑛 + 1) = 𝑤(𝑛) + 𝜇𝑥(𝑛)𝑒(𝑛) {𝛿 + (1 − 𝛿)(𝑒(𝑛)) } 0 ≤ 𝛿 ≤ 1 (𝐿𝑀𝑀𝑁) Affine projection 𝜇𝑤(𝑛 + 1) = 𝑤(𝑛) + 𝑥(𝑛)𝑒(𝑛) 𝐼 : Identity matrix LMS 𝑥(𝑛)𝑇𝑥(𝑛) + 𝜓𝐼 𝜓 : Small positive constant (𝐴𝑃𝐿𝑀𝑆) 4. NOISE CANCELLATION PROCESS WITH LMS Adaptive noise cancellation (ANC) is used in wide area to reduce noise from a corrupted signal. The aim of an ANC is to cancel the noise from a signal adaptively to improve the SNR. Fig. 3 shows the block diagram of adaptive noise cancellation. The system output in Fig. 3 is written by Eq. (12) (Widrow et al., 1975). 158 Uludağ University Journal of The Faculty of Engineering, Vol. 26, No. 1, 2021 Signal Primary input 𝒙 + 𝒏 𝟎 source  𝒙: Signal  𝒏𝟎: Noise uncorrelated with 𝒙  𝒏𝟏: Noise uncorrelated with 𝒙 𝒏 𝒚 but correlated with 𝒏Noise 𝟏 Adaptive 𝟎  𝒚: Filter output source Reference input filter + −  𝒆: Error 𝒆 𝒛 System  𝒛: System output output Adaptive noise canceller Figure 3: The block diagram of adaptive noise cancellation 𝑧 = 𝑥 + 𝑛0 − 𝑦 (12) If both sides of Eq. (12) are squared, Eq. (13) is obtained. 𝑧2 = 𝑥2 + 2𝑥(𝑛0 − 𝑦) + (𝑛 2 0 − 𝑦) (13) 𝐸{𝑧2} = 𝐸{𝑥2} + 2𝐸{𝑥(𝑛 − 𝑦)} + 𝐸{(𝑛 − 𝑦)2} = 𝐸{𝑥20 0 } + 𝐸{(𝑛0 − 𝑦) 2} (14) The minimum output power of system is 𝑚𝑖𝑛[𝐸{𝑧2}] = 𝐸{𝑥2} + 𝑚𝑖𝑛[𝐸{(𝑛0 − 𝑦) 2}] (15) In order to provide 𝐸{𝑧2} to be minimum, 𝐸{(𝑛0 − 𝑦) 2} must also be minimum in Eq. (15), hence 𝑧 − 𝑥 = 𝑛0 − 𝑦 (16) 5. DESIGNED APPLICATION In this study, an application/simulator was designed using MATLAB App Designer (MathWorks, 2018). With this application, noise cancellation operations can be performed with different variants of LMS algorithms. In the application, which includes information about algorithms, the effects of parameter changes can also be observed. In addition, single or comparative analyzes can be made on internal (sample or user-defined) and external (uploaded by the user) signals, and the results are displayed both numerically and graphically. Thus, the performance effects of both of the methods and the parameters related to the methods can be observed. In addition, it supports better understanding of both noise cancellation operations and LMS algorithms with its use in the field of education. The main screen of the designed application is shown in Fig. 4 and its menus are given in Table 4. The user/student can select the sample signal for analysis from the "Signal" drop-down list (combobox), generate different types of signals (generating sinusoidal, exponential, constant signals by entering their parameters) (Fig. 5a) or load signals from the file. Likewise, users can select the sample noise from the "Noise" drop-down list, generate noise themselves (generating noise by entering the scaling and shifting parameters in equation 𝑎{𝑟𝑎𝑛𝑑𝑜𝑚(0,1) − 𝑏}) (Fig. 5b) or load noise from the file. After each process, the related signal/noise graphics are simultaneously seen/plotted on the main screen. Finally, noise cancellation is performed by selecting LMS or one of its variants from the "Algorithm" drop-down list. If it is desired, comparative analysis can also be made with the "Comparative" option here. In addition, by selecting "Error analysis" or 159 Vatansever F.:Noise Cancellation with LMS Variants "Parameter analysis" from the menu that opens with the mouse right-click on the "de-noised signal" graph, error graphics/values and the effects of parameter changes can be viewed both numerically and graphically on the related screens. The flowchart of single analysis process in designed application is given in Fig. 6. Figure 4: The main screen of designed application 160 Uludağ University Journal of The Faculty of Engineering, Vol. 26, No. 1, 2021 Table 4. The menu items of designed application File Analysis Help (a) (b) Figure 5: The signal and noise generation screens Start Select the sample signal/ Generate signal / Load signal Select the sample noise/ Generate noise / Load noise Select the LMS algorithm Input the parameters of selected algorithm Analysis Show results in numerical and graphical format Stop Figure 6: The flowchart of single analysis process in designed application 161 Vatansever F.:Noise Cancellation with LMS Variants For example; when Signal-1 (𝑥 signal) and Noise-1 (𝑛0 noise) (Poularikas, 2015) items are selected, the screenshot in Fig. 7a is obtained. 𝜋 𝑥(𝑛) = 1.0𝑆𝑖𝑛 ( 𝑛) (17) 10 1 𝑛0(𝑛) = 2.0 (𝑟𝑎𝑛𝑑𝑜𝑚(0,1) − ) (18) 2 (a) (b) 162 Uludağ University Journal of The Faculty of Engineering, Vol. 26, No. 1, 2021 (c) When "Least Mean-Square (LMS)" is selected from the "Algorithm" drop-down list, the screen for setting the related algorithm parameters appears (Fig. 7b). After the related parameters are entered, the result screen in Fig. 7c is obtained. When the mouse is right-clicked on the "De-noise signal" graph and "Error analysis" is selected, the screen in Fig. 7d appears. This screen displays "learning curve", "absolute error" and "squared error" graphics and mean absolute error (MAE), mean squared error (MSE), sum absolute error (SAE) and sum squared error (SSE) performance values. In addition, by changing the starting point of the samples with related slider or spinner, the change of this error values can be monitored (Fig. 7e). On the other hand, if "Parameter analysis" is selected, comparative performance analysis can be performed by entering changeable parameter values of the related algorithm (Fig. 7f-g). In addition to the comparative performance graphics, the minimum values in the relevant error table are automatically colored on this screen. For "Comparative analysis", algorithms are selected on the algorithm selection screen, their parameters are set and the comparison screen in Fig. 7h appears. On this screen, the MSE values of each algorithm are listed in the table, and de-noised signals and error graphs are also plotted comparatively. Also, re-analysis can be made by changing the parameters (Fig. 7h). 163 Vatansever F.:Noise Cancellation with LMS Variants (d) (e) 164 Uludağ University Journal of The Faculty of Engineering, Vol. 26, No. 1, 2021 (f) (g) (h) Figure 7: Screenshots of the sample applications 165 Vatansever F.:Noise Cancellation with LMS Variants Summarization of comparison this study with similar studies in noise cancellation field are given below:  Generally, several methods or LMS variants are examined and performance comparisons are made in the studies. In the application developed in this study, there are many LMS variants, and single or comparative analysis can be performed.  In the studies, methods/methods are usually encoded in MATLAB environment (or ready-made functions are used) or SIMULINK models are prepared. In this study, an interactive interface is designed for the methods.  In some studies, hardware applications were carried out using FPGA or DSP kits. The hardware implementation is not included in the study in this article.  The general comparison of the simulator developed in the reference (Maddala, 2010) and the simulator designed in this study is given in Table 4. Table 4. The general comparison of designed application with other simulator Properties (Maddala, 2010) This study Used software MATLAB GUI MATLAB App Designer The number of methods Few Many Menus (open, save, print etc.) √ Method description √ Sample signals √ √ Generate signal √ Load signal √ Generate noise √ Load noise √ Other applications with LMS √ Parameters settings of methods √ √ Graphic of signal √ √ Graphic of learning curve √ Graphic of error √ √ Error values √ Error analysis √ Parameter analysis √ Comparative analysis √ The interactive application developed has features that can be used both in practice and education. For practical applications, the most appropriate algorithm and parameter values can be determined by analyzing the relevant noisy signals (by creating or loading) in the simulator. It can be used effectively in the field of education thanks to its capabilities such as including lectures, showing parameter effects in detail, and making comparisons. 6. CONCLUSIONS In this study, an application/simulator was carried out for noise cancellation, in one of the most important application areas of adaptive filters. With this application, which includes many variants of the LMS algorithm; the sample, user-defined or loading noisy signals can be de-noised easily and quickly with the selected algorithm or algorithms. It is obvious that LMS algorithms and noise cancellation processes can be better understood by using the developed interactive application especially in the field of engineering education. The ability of the simulator to give opportunity to students/users to examine lectures, change algorithm parameters, and observe single or comparative results both numerically and graphically will contribute greatly to learning and comprehension. As a future study, it is aimed to add more LMS variants to the application/simulator and move it to the web environment. 166 Uludağ University Journal of The Faculty of Engineering, Vol. 26, No. 1, 2021 CONFLICT OF INTEREST Author approves that to the best of his knowledge, there is not any conflict of interest or common interest with an institution/organization or a person that may affect the review process of the paper. AUTHOR CONTRIBUTION Fahri Vatansever has full responsibility for all parts of the paper. REFERENCES 1. Bismor, D., Czyz, K., Ogonowski, Z. (2016) Review and comparison of variable step-size LMS algorithms, The International Journal of Acoustics and Vibration, 21, 24-39. doi: https://doi.org/10.20855/ijav.2016.21.1392. 2. Diniz, P.S.R. (2020) Adaptive Filtering: Algorithms and Practical Implementation, 5th ed., Springer, Switzerland. 3. Dixit, S., Nagaria, D. (2017) LMS adaptive filters for noise cancellation: A review, International Journal of Electrical and Computer Engineering, 7(5), 2520-2529. doi: https://doi.org/10.11591/ijece.v7i5.pp2520-2529 4. Farhang-Boroujeny, B. (2013) Adaptive Filters: Theory and Applications, 2nd ed., Wiley. 5. Goel, P., Chandra, M. (2019) FPGA implementation of adaptive filtering algorithms for noise cancellation - A Technical survey, In: Nath, V., Mandal, J. (eds) Proceedings of the Third International Conference on Microelectronics, Computing and Communication Systems. Lecture Notes in Electrical Engineering, 556, Springer, Singapore. https://doi.org/10.1007/978-981-13-7091-5_42. 6. Gorriz, J.M, Ramirez, J., Cruces-Alvarez, S., Puntonet, C.G., Lang, E.W., Erdogmus, D. (2009) A Novel LMS algorithm applied to adaptive noise cancellation, IEEE Signal Processing Letters, 16(1), 34-37. doi: https://doi.org/10.1109/LSP.2008.2008584. 7. Guda, M., Gasser, S., El Mahallawy, M.S. (2014) MATLAB simulation comparison for different adaptive noise cancelation algorithms, The International Conference on Digital Information, Networking, and Wireless Communications (DINWC), Czech Republic, 68-73. 8. Hatun, M., Vatansever, F. (2016) Differential equation solver simulator for Runge-Kutta methods, Uludağ University Journal of The Faculty of Engineering, 21(1), 147-164. doi: https://doi.org/10.17482/uujfe.70981. 9. Haykin, S. (2014) Adaptive Filter Theory, 5th ed., Pearson, USA. 10. Kaçar, S., Çankaya, İ. (2010) Volterra serileri metodu ile doğrusal olmayan sistemlerin frekans boyutunda analizi için .NET tabanlı arayüz tasarımı, Dokuz Eylül Üniversitesi Mühendislik Fakültesi Fen ve Mühendislik Dergisi, 12(3), 87-102. 11. Kaçar, S., Çankaya, İ. (2012) MATLAB ve ASP.NET tabanlı web arayüzü kullanılarak doğrusal olmayan sistemlerin analizi, Gazi Üniversitesi Mühendislik Mimarlık Fakültesi Dergisi, 27(4), 795-806. 12. Kaçar, S., Boz, A.F., Arıcıoğlu, B., Tekin, H. (2017) PID denetleyici uygulamaları için yeni bir online deney seti tasarımı, Sakarya Üniversitesi Fen Bilimleri Enstitüsü Dergisi, 21(1), 34-46. doi: https://doi.org/10.16984/saufenbilder.284220. 167 Vatansever F.:Noise Cancellation with LMS Variants 13. Lee, J-H., Ooi, L-E., Ko, Y-H., Teoh, C-Y. (2017) Simulation for noise cancellation using LMS adaptive filter, IOP Conf. Series: Materials Science and Engineering, 211, 012003, doi: https://doi.org/10.1088/1757-899X/211/1/012003. 14. Maddala, S. (2010) A simulator for depicting and comparing adaptive algorithms in signal processing, 2010 International Conference on Signal and Image Processing, Chennai, 515- 520. doi: https://doi.org/10.1109/ICSIP.2010.5697530. 15. Maurya, A.K. (2018) Cascade–cascade least mean square (LMS) adaptive noise cancellation, Circuits Syst Signal Process, 37, 3785–3826. doi: https://doi.org/10.1007/s00034-017-0731- 2. 16. Maurya, A.K., Agrawal, P., Dixit, S. (2019) Modified model and algorithm of LMS adaptive filer for noise cancellation, Circuits Syst Signal Process, 38, 2351-2368. doi: https://doi.org/10.1007/s00034-018-0952-z. 17. Morales, L.G. (Ed.) (2011) Adaptive Filtering Applications, InTech, Croatia. 18. Poularikas, A.D. (2015) Adaptive Filtering, CRC Press. 19. Romero, I., Geng, D., Berset, T. (2012) Adaptive filtering in ECG denoising: A comparative study, 2012 Computing in Cardiology, Krakow, 45-48. 20. Qureshi, R., Rizvi, S.A.R., Musavi, S.H.A., Khan, S., Khurshid, K. (2017) Performance analysis of adaptive algorithms for removal of low frequency noise from ECG signal, 2017 International Conference on Innovations in Electrical Engineering and Computational Technologies (ICIEECT), Karachi, 1-5. doi: https://doi.org/10.1109/ICIEECT.2017.7916551. 21. Sankar, A.B., Kumar, D., Seethalakshmi, K. (2010) Performance study of various adaptive filter algorithms for noise cancellation in respiratory signals, Signal Processing: An International Journal (SPIJ), 4(5), 267-278. 22. Sayed, A.H. (2008) Adaptive Filters, Wiley, USA. 23. Taha, T.M.F., Adeel, A., Hussain, A. (2018) A Survey on techniques for enhancing speech, International Journal of Computer Applications, 179(17), 1-14. doi: https://doi.org/10.5120/ijca2018916290. 24. Thakor, N.V., Zhu, Y-S. (1991) Applications of adaptive filtering to ECG analysis: noise cancellation and arrhythmia detection, IEEE Transactions on Biomedical Engineering, 38(8), 785-794. doi: https://doi.org/10.1109/10.83591. 25. The MathWorks Inc. (2018) MATLAB , App Designer. 26. Vatansever, F., Yalcin, N.A. (2017) e-Signals & Systems: A web-based educational tool for signals and systems, Computer Applications in Engineering Education, 25(4), 625-641. doi: https://doi.org/10.1002/cae.21826. 27. Vatansever, F., Hatun, M. (2019) The system stability software tool based on Routh-Hurwitz criterion, Uludağ University Journal of The Faculty of Engineering, 24(2), 229-238. doi: https://doi.org/10.17482/uumfd.545361. 28. Yalcin, N.A., Vatansever, F. (2016) A Web-based virtual power electronics laboratory, Comput. Appl. Eng. Educ., 24(1), 71-78. doi: https://doi.org/10.1002/cae.21673. 29. Widrow, B., Glover, J.R., McCool, J.M., Kaunitz, J., Williams, C.S., Hearn, R.H., Zeidler, J.R., Donc, E., Goodlin, R.C. (1975) Adaptive noise cancelling: Principles and applications, Proceedings of the IEEE, 63(12), 1692-1716. doi: https://doi.org/10.1109/PROC.1975.10036. 168 Uludağ University Journal of The Faculty of Engineering, Vol. 26, No. 1, 2021 30. Widrow, B., Stearns, S.D. (1985) Adaptive Signal Processing, Prentice-Hall, USA. 31. Zaknich, A. (2005) Principles of Adaptive Filters and Self-learning Systems, Springer, Germany. 169 170