8338197: ubsan: ad_x86.hpp:6417:11: runtime error: shift exponent 100 is too large for 32-bit type 'unsigned int'

[bulasevich]

This reworks the recent update #24696 to fix a UBSan issue on aarch64. The problem now reproduces on x86_64 as well, which suggests the previous update was not optimal.

The issue reproduces with a HeapByteBufferTest jtreg test on a UBSan-enabled build. Actually the trigger is XX:+OptoScheduling option used by test (by default OptoScheduling is disabled on most x86 CPUs). With the option enabled, the failure can be reproduced with a simple java -version run.

This fix is in ADLC-generated code. For simplicity, the examples below show the generated fragments.

The problems is that shift count n may be too large here:

class Pipeline_Use_Cycle_Mask {
protected:
  uint _mask;
  ..
  Pipeline_Use_Cycle_Mask& operator<<=(int n) {
    _mask <<= n;
    return *this;
  }
};

The recent change attempted to cap the shift amount at one call site:

class Pipeline_Use_Element {
protected:
  ..
  // Mask of specific used cycles
  Pipeline_Use_Cycle_Mask _mask;
  ..
  void step(uint cycles) {
    _used = 0;
    uint max_shift = 8 * sizeof(_mask) - 1;
    _mask <<= (cycles < max_shift) ? cycles : max_shift;
  }
}

However, there is another site where Pipeline_Use_Cycle_Mask::operator<<= can be called with a too-large shift count:

// The following two routines assume that the root Pipeline_Use entity
// consists of exactly 1 element for each functional unit
// start is relative to the current cycle; used for latency-based info
uint Pipeline_Use::full_latency(uint delay, const Pipeline_Use &pred) const {
  for (uint i = 0; i < pred._count; i++) {
    const Pipeline_Use_Element *predUse = pred.element(i);
    if (predUse->_multiple) {
      uint min_delay = 7;
      // Multiple possible functional units, choose first unused one
      for (uint j = predUse->_lb; j <= predUse->_ub; j++) {
        const Pipeline_Use_Element *currUse = element(j);
        uint curr_delay = delay;
        if (predUse->_used & currUse->_used) {
          Pipeline_Use_Cycle_Mask x = predUse->_mask;
          Pipeline_Use_Cycle_Mask y = currUse->_mask;

          for ( y <<= curr_delay; x.overlaps(y); curr_delay++ )
            y <<= 1;
        }
        if (min_delay > curr_delay)
          min_delay = curr_delay;
      }
      if (delay < min_delay)
      delay = min_delay;
    }
    else {
      for (uint j = predUse->_lb; j <= predUse->_ub; j++) {
        const Pipeline_Use_Element *currUse = element(j);
        if (predUse->_used & currUse->_used) {
          Pipeline_Use_Cycle_Mask x = predUse->_mask;
          Pipeline_Use_Cycle_Mask y = currUse->_mask;

>        for ( y <<= delay; x.overlaps(y); delay++ )
            y <<= 1;
        }
      }
    }
  }

  return (delay);
}

Fix: cap the shift inside Pipeline_Use_Cycle_Mask::operator<<= so all call sites are safe:

class Pipeline_Use_Cycle_Mask {
protected:
  uint _mask;
  ..
  Pipeline_Use_Cycle_Mask& operator<<=(int n) {
    int max_shift = 8 * sizeof(_mask) - 1;
    _mask <<= (n < max_shift) ? n : max_shift;
    return *this;
  }
};

class Pipeline_Use_Element {
protected:
  ..
  // Mask of specific used cycles
  Pipeline_Use_Cycle_Mask _mask;
  ..
  void step(uint cycles) {
    _used = 0;
    _mask <<= cycles;
  }
}

Note: on platforms where PipelineForm::_maxcycleused > 32 (e.g., ARM32), the Pipeline_Use_Cycle_Mask implementation already handles large shifts, so no additional check is needed:

class Pipeline_Use_Cycle_Mask {
protected:
  uint _mask1, _mask2, _mask3;

  Pipeline_Use_Cycle_Mask& operator<<=(int n) {
    if (n >= 32)
      do {
        _mask3 = _mask2; _mask2 = _mask1; _mask1 = 0;
      } while ((n -= 32) >= 32);

    if (n > 0) {
      uint m = 32 - n;
      uint mask = (1 << n) - 1;
      uint temp2 = mask & (_mask1 >> m); _mask1 <<= n;
      uint temp3 = mask & (_mask2 >> m); _mask2 <<= n; _mask2 |= temp2;
      _mask3 <<= n; _mask3 |= temp3;
    }
    return *this;
  }
}

---

Progress

• Change must be properly reviewed (1 review required, with at least 1 Reviewer)
• Change must not contain extraneous whitespace
• Commit message must refer to an issue

Issue

• JDK-8338197: ubsan: ad_x86.hpp:6417:11: runtime error: shift exponent 100 is too large for 32-bit type 'unsigned int' (Bug - P4)

Reviewing

Using git

Checkout this PR locally:
$ git fetch https://git.openjdk.org/jdk.git pull/26890/head:pull/26890
$ git checkout pull/26890

Update a local copy of the PR:
$ git checkout pull/26890
$ git pull https://git.openjdk.org/jdk.git pull/26890/head

Using Skara CLI tools

Checkout this PR locally:
$ git pr checkout 26890

View PR using the GUI difftool:
$ git pr show -t 26890

Using diff file

Download this PR as a diff file:
https://git.openjdk.org/jdk/pull/26890.diff

Using Webrev

Link to Webrev Comment

[bridgekeeper]

👋 Welcome back bulasevich! A progress list of the required criteria for merging this PR into master will be added to the body of your pull request. There are additional pull request commands available for use with this pull request.

[openjdk]

❗ This change is not yet ready to be integrated.
See the Progress checklist in the description for automated requirements.

[openjdk]

@bulasevich The following label will be automatically applied to this pull request:

• hotspot-compiler

When this pull request is ready to be reviewed, an "RFR" email will be sent to the corresponding mailing list. If you would like to change these labels, use the /label pull request command.

[mlbridge]

Webrevs

• 01: Full - Incremental (389a9dab)
• 00: Full (7e8c282d)

[vnkozlov]

sizeof(_mask) is know - it is sizeof(uint).
Lines 760-768 should be cleaned: <= 32 checks are redundant because of check at line 758. This is leftover from SPARC code (not clean) removal.

[bulasevich]

Good point - I removed the redundant code.

As for sizeof(_mask), shouldn’t it just be max_shift = 31 or _mask <<= (n < 32) ? n : 31;?

[vnkozlov]

Yes, if sizeof(uint) is 32 bits on all our platforms.

Hmm, may be we should use uint32_t for _mask here. Then we can use 32 and 31 without confusion.

[vnkozlov]

I mean to use _mask <<= (n < 32) ? n : 31;

[dean-long]

I didn't realize we already had code to handle masks for large shifts. So I think the main problem is that _maxcycleused is not being set to the max value of 100. There is a secondary problem that we don't really need values that high, if the units are in pipeline stages.