Add 'Longest Increasing Subsequence'

Author: nickolashkraus

README.md

@@ -24,6 +24,8 @@ A collection of LeetCode solutions
 
 [Longest Common Subsequence](./src/longest_common_subsequence.py)
 
+[Longest Increasing Subsequence](./src/longest_increasing_subsequence.py)
+
 [Maximum Depth of Binary Tree](./src/maximum_depth_of_binary_tree.py)
 
 [Maximum Subarray](./src/maximum_subarray.py)

TODO.md

@@ -5,3 +5,5 @@
 - [x] Add unit tests
 - [ ] Solve 'Same Tree' using recursion (DFS)
 - [ ] Review other approaches to solving 'Subtree of Another Tree'
+- [ ] Add alternative solution for 'Longest Increasing Subsequence'
+- [ ] Add binary search solution for 'Longest Increasing Subsequence'

pyproject.toml

@@ -30,6 +30,7 @@ dependencies = [
 # RET505: Unnecessary `elif` after `return` statement
 # RUF002: Docstring contains ambiguous `ν`
 # RUF003: Docstring contains ambiguous `ν`
+# SIM108 Use ternary operator
 # T201: `print` found
 # W605: invalid escape sequence '\ '
-ignore = ["A002", "E741", "N802", "N803", "N806", "N999", "RET505", "RUF002", "RUF003", "T201", "W605"]
+ignore = ["A002", "E741", "N802", "N803", "N806", "N999", "RET505", "RUF002", "RUF003", "SIM108", "T201", "W605"]

src/longest_increasing_subsequence.py

@@ -4,19 +4,42 @@
 https://leetcode.com/problems/longest-increasing-subsequence
 
 NOTES
+  * Use dynamic programming (1D) or recursion.
 
-My initial thought is this is a 1D dynamic programming problem, since the
-problem can be built up from its subproblems.
+My initial intuition was this is a dynamic programming (1D) problem, since the
+problem can be built up from solutions to its subproblems. However, I was
+unable to formulate the correct recurrence relation. I was, however, able to
+derive the correct recursive solution.
 
 As an initial approach, let's find the base cases:
 
-  1. If nums = [], return 0
-  2. If nums has the length 1, return 1
+  1. If nums is empty, return 0
+  2. If nums has length 1, return 1
 
 Next, we look that a simplified example:
 
-  3. If nums = [1, 2], (i + 1) > 1, therefore return 1 + 1
-  4. If nums = [1, 3, 2, 4], return 1 + lis([3, 2, 4]) (which is 2)
+  3. If nums = [1, 2],
+     2 > 1, therefore LIS = 2
+
+  4. If nums = [1, 5, 2, 4],
+     5 > 1, therefore LIS([1,5]) = 2, however, LIS([1, 5, 2, 4]) = 3
+
+How should we account for the fact that the longest increasing subsequence is
+formed not by using 5, but instead by using 2 and 4? Programmatically, this
+signifies a decision branch that can be resolved recursively:
+
+  1. When nums[i] is included in the subsequence
+  2. When nums[i] is not included in the subsequence
+
+For the first case, we increment the LIS by 1 and update the current largest
+element in the subsequence. For the second case, we simply ignore nums[i],
+retaining the current largest element in the subsequence. We then take the
+maximum of each branch of the decision tree.
+
+    if nums[0] > m:
+        return max(1 + lis(nums[1:], nums[0]), lis(nums[1:], m))
+    else:
+        return lis(nums[1:], m)
 
 Realizing a Dynamic Programming Problem
 ---------------------------------------
@@ -27,21 +50,29 @@
 previously made decisions, which is very typical of a problem involving
 subsequences.
 
+As we go through the input, each "decision" we must make is simple: is it worth
+it to consider this number? If we use a number, it may contribute towards an
+increasing subsequence, but it may also eliminate larger elements that came
+before it. For example, let's say we have nums = [5, 6, 7, 8, 1, 2, 3]. It
+isn't worth using the 1, 2, or 3, since using any of them would eliminate 5, 6,
+7, and 8, which form the longest increasing subsequence. We can use dynamic
+programming to determine whether an element is worth using or not.
+
 A Framework to Solve Dynamic Programming Problems
 -------------------------------------------------
 
 Typically, dynamic programming problems can be solved with three main
 components. If you're new to dynamic programming, this might be hard to
-understand, but it is extremely valuable to learn, since most dynamic
-programming problems can be solved this way.
+understand but is extremely valuable to learn since most dynamic programming
+problems can be solved this way.
 
 First, we need some function or array that represents the answer to the problem
-from a given state. For many solutions on LeetCode, you will see this
-function/array named "dp". For this problem, let's say that we have an array
-dp. As previously stated, this array needs to represent the answer to the
-problem for a given state, so let's say that dp[i] represents the length of the
-longest increasing subsequence that ends with the ith element. The "state" is
-one-dimensional since it can be represented with only one variable–the index i.
+from a given state. Typically, since array is named "dp". For this problem,
+let's say that we have an array dp. As just stated, this array needs to
+represent the answer to the problem for a given state, so let's say that dp[i]
+represents the length of the longest increasing subsequence that ends with the
+ith element. The "state" is one-dimensional since it can be represented with
+only one variable - the index i.
 
 Second, we need a way to transition between states, such as dp[5] and dp[7].
 This is called a recurrence relation and can sometimes be tricky to figure out.
@@ -57,14 +88,6 @@
 The third component is the simplest: we need a base case. For this problem, we
 can initialize every element of dp to 1, since every element on its own is
 technically an increasing subsequence.
-
-NOTE: I got close when trying to solve this with recursion and memoization,
-however my solutions failed the following test case:
-
-  [3, 5, 6, 2, 5, 4, 19, 5, 6, 7, 12]
-
-ChatGippity could not solve it either, so this can be a future problem to
-return to...
 """
 
 import sys
@@ -84,62 +107,71 @@ def lengthOfLIS(self, nums: list[int]) -> int:
 
 class MemoizationSolution:
     """
-    Similar to the top-down approach of the recursive solution, but uses
-    memoization to store previous calculations.
+    Similar to the recursive solution, but uses memoization to store previous
+    calculations. This one is a little bit tricky, since the memoization lookup
+    uses both the index and m, the current largest element in the subsequence.
 
-    NOTE: This solution fails the following test case:
-
-      [3, 5, 6, 2, 5, 4, 19, 5, 6, 7, 12] -> 5 == 3
+    NOTE: Though *technically* correct, this solution still exceeds the time
+    limit.
     """
 
     def lengthOfLIS(self, nums: list[int]) -> int:
-        # Initialize the memoization table. Each index in the table is the
-        # calculated longest increasing subsequence up to and including the
-        # current index.
-        memo: list[int] = [1] * len(nums)
-
-        def lis(nums: list[int], m: int, memo) -> int:
-            if len(nums) == 1:
-                return 1 if nums[0] > m else 0
+        # Initialize the memoization table. Each key in the table is formed
+        # using an index in nums and the current largest element in the
+        # subsequence. The stored value is the current longest increasing
+        # subsequence for the index.
+        #
+        # One effect of the using the index and current largest element for the
+        # key is that it actually lets us handle overlapping subsequences
+        # correctly, since we're caching results for each unique combination of
+        # remaining numbers and minimum value.
+        memo: dict[tuple[int, int], int] = {}
+
+        def lis(nums: list[int], i: int, m: int) -> int:
+            if i >= len(nums):
+                return 0
+            key = (i, m)
+            if key in memo:
+                return memo[key]
             # Calculate the result of each decision:
             #
             #   1. nums[i] is included in the subsequence
-            #   2. nums[i] is *not* included in the subsequence
-            i = len(memo) - len(nums)
-            if memo[i] > 1:
-                return memo[i]
-            if nums[0] > m:
-                memo[i] = max(1 + lis(nums[1:], nums[0], memo), lis(nums[1:], m, memo))
+            #   2. nums[i] is not included in the subsequence
+            #
+            # nums[i] becomes the current largest element in the subsequence.
+            if nums[i] > m:
+                return max(lis(nums, i + 1, nums[i]) + 1, lis(nums, i + 1, m))
             else:
-                memo[i] = lis(nums[1:], m, memo)
-            return memo[i]
+                res = lis(nums, i + 1, m)
+            memo[key] = res
+            return res
 
-        # NOTE: -10^4 <= nums[i] <= 10^4
-        return lis(nums, -sys.maxsize, memo)
+        return lis(nums, 0, -sys.maxsize)
 
 
 class RecursiveSolution:
     """
-    NOTE: This solution exceeds the time limit.
+    The recursive solution to Longest Increasing Subsequence uses the intuition
+    that an element should be added to the subsequence if it is greater than
+    the current largest element in the subsequence.
 
-    NOTE: This solution fails the following test case:
-
-      [3, 5, 6, 2, 5, 4, 19, 5, 6, 7, 12] -> 6 == 3
+    NOTE: Though *technically* correct, this solution exceeds the time limit,
+    since it does not account for overlapping subproblems.
     """
 
     def lengthOfLIS(self, nums: list[int]) -> int:
-        def lis(nums: list[int], m: int) -> int:
-            if len(nums) == 1:
-                return 1 if nums[0] > m else 0
-
+        def lis(nums: list[int], i: int, m: int) -> int:
+            if i >= len(nums):
+                return 0
             # Calculate the result of each decision:
             #
             #   1. nums[i] is included in the subsequence
-            #   2. nums[i] is *not* included in the subsequence
-            if nums[0] > m:
-                return max(1 + lis(nums[1:], nums[0]), lis(nums[1:], m))
+            #   2. nums[i] is not included in the subsequence
+            #
+            # nums[i] becomes the current largest element in the subsequence.
+            if nums[i] > m:
+                return max(lis(nums, i + 1, nums[i]) + 1, lis(nums, i + 1, m))
             else:
-                return lis(nums[1:], m)
+                return lis(nums, i + 1, m)
 
-        # NOTE: -10^4 <= nums[i] <= 10^4
-        return lis(nums, -sys.maxsize)
+        return lis(nums, 0, -sys.maxsize)

tests/test_longest_increasing_subsequence.py

@@ -6,25 +6,29 @@
 
 from unittest import TestCase
 
-from src.longest_increasing_subsequence import MemoizationSolution, RecursiveSolution
+from src.longest_increasing_subsequence import MemoizationSolution, RecursiveSolution, Solution
 
 
 class TestSolution(TestCase):
     def test_1(self):
         exp = 4
-        assert MemoizationSolution().lengthOfLIS([10, 9, 2, 5, 3, 7, 101, 18]) == exp
+        assert Solution().lengthOfLIS([10, 9, 2, 5, 3, 7, 101, 18]) == exp
 
     def test_2(self):
         exp = 4
-        assert MemoizationSolution().lengthOfLIS([0, 1, 0, 3, 2, 3]) == exp
+        assert Solution().lengthOfLIS([0, 1, 0, 3, 2, 3]) == exp
 
     def test_3(self):
         exp = 1
-        assert MemoizationSolution().lengthOfLIS([7, 7, 7, 7, 7, 7, 7]) == exp
+        assert Solution().lengthOfLIS([7, 7, 7, 7, 7, 7, 7]) == exp
 
     def test_4(self):
         exp = 3
-        assert MemoizationSolution().lengthOfLIS([4, 10, 4, 3, 8, 9]) == exp
+        assert Solution().lengthOfLIS([4, 10, 4, 3, 8, 9]) == exp
+
+    def test_5(self):
+        exp = 6
+        assert Solution().lengthOfLIS([3, 5, 6, 2, 5, 4, 19, 5, 6, 7, 12]) == exp
 
 
 class TestMemoizationSolution(TestCase):
@@ -44,13 +48,9 @@ def test_4(self):
         exp = 3
         assert MemoizationSolution().lengthOfLIS([4, 10, 4, 3, 8, 9]) == exp
 
-    # NOTE: Implementation fails the following testcase:
-    # def test_5(self):
-    #     exp = 3
-    #     assert (
-    #         MemoizationSolution().lengthOfLIS([3, 5, 6, 2, 5, 4, 19, 5, 6, 7, 12])
-    #         == exp
-    #     )
+    def test_5(self):
+        exp = 6
+        assert MemoizationSolution().lengthOfLIS([3, 5, 6, 2, 5, 4, 19, 5, 6, 7, 12]) == exp
 
 
 class TestRecursiveSolution(TestCase):
@@ -70,9 +70,6 @@ def test_4(self):
         exp = 3
         assert RecursiveSolution().lengthOfLIS([4, 10, 4, 3, 8, 9]) == exp
 
-    # NOTE: Implementation fails the following testcase:
-    # def test_5(self):
-    #     exp = 3
-    #     assert (
-    #         RecursiveSolution().lengthOfLIS([3, 5, 6, 2, 5, 4, 19, 5, 6, 7, 12]) == exp
-    #     )
+    def test_5(self):
+        exp = 6
+        assert RecursiveSolution().lengthOfLIS([3, 5, 6, 2, 5, 4, 19, 5, 6, 7, 12]) == exp