Merge pull request Open-Deep-ML#274 from Haleshot/haleshot/hard-sigmoid

New Problem: Hard Sigmoid

Author: moe18

Problems/96_Hard_Sigmoid/learn.md

@@ -0,0 +1,33 @@
+## Understanding the Hard Sigmoid Activation Function
+
+The Hard Sigmoid is a piecewise linear approximation of the sigmoid activation function. It's computationally more efficient than the standard sigmoid function while maintaining similar characteristics. This function is particularly useful in neural networks where computational efficiency is crucial.
+
+### Mathematical Definition
+
+The Hard Sigmoid function is mathematically defined as:
+
+$$
+HardSigmoid(x) = \begin{cases} 
+0 & \text{if } x \leq -2.5 \\ 
+0.2x + 0.5 & \text{if } -2.5 < x < 2.5 \\ 
+1 & \text{if } x \geq 2.5 
+\end{cases}
+$$
+
+Where $x$ is the input to the function.
+
+### Characteristics
+
+- **Output Range:** The output is always bounded in the range $[0, 1]$
+- **Shape:** The function consists of three parts:
+  - A constant value of 0 for inputs ≤ -2.5
+  - A linear segment with slope 0.2 for inputs between -2.5 and 2.5
+  - A constant value of 1 for inputs ≥ 2.5
+- **Gradient:** The gradient is 0.2 in the linear region and 0 in the saturated regions
+
+### Advantages in Neural Networks
+
+This function is particularly useful in neural networks as it provides:
+- Computational efficiency compared to standard sigmoid
+- Bounded output range similar to sigmoid
+- Simple gradient computation

Problems/96_Hard_Sigmoid/solution.py

@@ -0,0 +1,36 @@
+def hard_sigmoid(x: float) -> float:
+    """
+    Implements the Hard Sigmoid activation function.
+    
+    Args:
+        x (float): Input value
+        
+    Returns:
+        float: The Hard Sigmoid of the input
+    """
+    if x <= -2.5:
+        return 0.0
+    elif x >= 2.5:
+        return 1.0
+    else:
+        return 0.2 * x + 0.5
+
+def test_hard_sigmoid():
+    # Test case 1: x <= -2.5
+    assert hard_sigmoid(-3.0) == 0.0, "Test case 1 failed"
+    
+    # Test case 2: x >= 2.5
+    assert hard_sigmoid(3.0) == 1.0, "Test case 2 failed"
+    
+    # Test case 3: -2.5 < x < 2.5
+    assert abs(hard_sigmoid(0.0) - 0.5) < 1e-6, "Test case 3 failed"
+    assert abs(hard_sigmoid(1.0) - 0.7) < 1e-6, "Test case 4 failed"
+    assert abs(hard_sigmoid(-1.0) - 0.3) < 1e-6, "Test case 5 failed"
+    
+    # Test boundary cases
+    assert abs(hard_sigmoid(2.5) - 1.0) < 1e-6, "Test case 6 failed"
+    assert abs(hard_sigmoid(-2.5) - 0.0) < 1e-6, "Test case 7 failed"
+
+if __name__ == "__main__":
+    test_hard_sigmoid()
+    print("All Hard Sigmoid tests passed.")