[Solved] SQL Query extremely slow when including NOT LIKE or a current row reference in a subquery

How to optimize this SQL query?

In case you have your own slow SQL query, you can optimize it automatically here.

For the query above, the following recommendations will be helpful as part of the SQL tuning process.
You'll find 3 sections below:

  1. Description of the steps you can take to speed up the query.
  2. The optimal indexes for this query, which you can copy and create in your database.
  3. An automatically re-written query you can copy and execute in your database.
The optimization process and recommendations:
  1. Avoid LIKE Searches With Leading Wildcard (query line: 30): The database will not use an index when using like searches with a leading wildcard (e.g. '%MAC%'). Although it's not always a satisfactory solution, please consider using prefix-match LIKE patterns (e.g. 'TERM%').
  2. Avoid LIKE Searches With Leading Wildcard (query line: 31): The database will not use an index when using like searches with a leading wildcard (e.g. '%Maint%'). Although it's not always a satisfactory solution, please consider using prefix-match LIKE patterns (e.g. 'TERM%').
  3. Avoid LIKE Searches With Leading Wildcard (query line: 35): The database will not use an index when using like searches with a leading wildcard (e.g. '%Closed - Won%'). Although it's not always a satisfactory solution, please consider using prefix-match LIKE patterns (e.g. 'TERM%').
  4. Create Optimal Indexes (modified query below): The recommended indexes are an integral part of this optimization effort and should be created before testing the execution duration of the optimized query.
  5. Index Function Calls Using Generated Columns (modified query below): When a function is used directly on an indexed column, the database's optimizer won’t be able to use the index to optimize the search. Creating and indexing a generated column (supported in MySQL 5.7) will allow MySQL to optimize the search.
  6. Prefer Inner Join Over Left Join (modified query below): We identified that one or more left joined entities (e.g. `SO_Type`) are used in the 'where' clause, in a way that allows to replace it with an optimized inner join. Inner joins can be fully optimized by the database, while Left joins apply limitations on the database's optimizer.
  7. Prefer Inner Join Over Left Join (modified query below): We identified that one or more left joined entities (e.g. `SO_Opportunity_Audit_Value`) are used in the 'where' clause, in a way that allows to replace it with an optimized inner join. Inner joins can be fully optimized by the database, while Left joins apply limitations on the database's optimizer.
Optimal indexes for this query:
CREATE INDEX so_dtl_idx_opportunity_recid ON SO_Forecast_dtl (Opportunity_RecID);
CREATE INDEX so_opportunity_idx_so_recid_opportunity_rec ON SO_Opportunity (SO_Opp_Status_RecID,Opportunity_RecID);
CREATE INDEX so_audit_idx_year_utc_so_recid_month_utc ON SO_Opportunity_Audit (year_soa_last_updated_utc,SO_Opportunity_Audit_RecId,month_soa_last_updated_utc);
CREATE INDEX so_audit_idx_audit_token ON SO_Opportunity_Audit_Value (audit_token);
CREATE INDEX so_type_idx_so_recid ON SO_Type (SO_Type_RecID);
CREATE INDEX sales_quarterly_idx_fore_year_fore_loc_fore_month ON dbo.sales_forecast_quarterly (fore_quart_year,fore_quart_loc,fore_quart_month);
The optimized query:
SELECT
        DateName(month,
        DateAdd(month,
        [sfq].[fore_quart_month],
        -1)) AS [Month],
        [sfq].[fore_quart_so_rev] AS [Sales Orders Revenue],
        [sfq].[fore_quart_so_mar] AS [Sales Orders Margin],
        [sfq].[fore_quart_mac_rev] AS [MAC Revenue],
        [sfq].[fore_quart_mac_mar] AS [MAC Margin],
        [sfq].[fore_quart_total_rev] AS [TOTAL Revenue],
        [sfq].[fore_quart_total_mar] AS [TOTAL Margin],
        (SELECT
            SUM([FORE].[Revenue]) 
        FROM
            [SO_Opportunity] [SO] 
        INNER JOIN
            [SO_Type] 
                ON [SO].[SO_Type_RecID] = [SO_Type].[SO_Type_RecID] 
        LEFT JOIN
            [SO_Opportunity_Audit] [soa] 
                ON [so].[Opportunity_RecID] = [soa].[Opportunity_RecId] 
        INNER JOIN
            [SO_Opportunity_Audit_Value] [soav] 
                ON [soa].[SO_Opportunity_Audit_RecId] = [soav].[SO_Opportunity_audit_recid] 
        LEFT JOIN
            [SO_Forecast_dtl] [FORE] 
                ON [SO].[Opportunity_RecID] = [FORE].[Opportunity_RecID] 
        WHERE
            (
                [SO_Type].[Description] NOT LIKE '%MAC%' 
                AND [SO_Type].[Description] NOT LIKE '%Maint%'
            ) 
            AND [soa].year_soa_last_updated_utc = @p_year 
            AND [soa].month_soa_last_updated_utc = [sfq].[fore_quart_month] 
            AND [soav].[audit_value] LIKE '%Closed - Won%' 
            AND [soav].[audit_token] = 'new_value' 
            AND [so].[SO_Opp_Status_RecID] = 7) AS [Rev] 
    FROM
        [authmanager2].[dbo].[sales_forecast_quarterly] [sfq] 
    WHERE
        [sfq].[fore_quart_year] = @p_year 
        AND [sfq].[fore_quart_loc] = 'w' 
    ORDER BY
        [sfq].[fore_quart_month]

Related Articles



* original question posted on StackOverflow here.