Commit Graph

1058 Commits

Author SHA1 Message Date
Adrian Kuegel 5315997402 Fix Sinh approximation for F16.
We should upcast F16 to F32 to prevent precision loss.
E.g. sinh(-9) would evaluate to -4042 previously instead of -4052.
This allows to enable the MLIR generated kernel for F16 type.

PiperOrigin-RevId: 377901896
2021-06-07 06:38:42 -07:00
Tobias Gysi fc723380e6 Update lhlo to use the new structured op interface.
Replace deprecated methods in lhlo_fuse_linalg.cc. The new structured op interface has been introduced in https://reviews.llvm.org/D103394.

PiperOrigin-RevId: 377875452
2021-06-07 03:11:03 -07:00
Wenyi Zhao ade873a5e0 PR #49970: [MLIR][DISC] bufferize DynamicReshape and DynamicBroadcastInDim
Imported from GitHub PR https://github.com/tensorflow/tensorflow/pull/49970

1, add hlo-to-lhlo support for DynamicReshape and DynamicBroadcastInDim

2, add a flag `convert-to-lmhlo-only` to seperate following two case:
   - hlo-to-lhlo only. Simply lowers all mhlo ops to their lmhlo
     counterparts, do not apply any optimization (e.g. elide any
     buffer copy). Buffer optimization is not easy in dynamic
     shape world especially when involving control flow, thus we
     leave this to another dedicated pass.

   - hlo-to-lhlo-or-memref-directly. Lowers some metadata-only mhlo
     ops (e.g. reshape) to memref dialect directly and Lowers others
     to their lmhlo counterparts.
Copybara import of the project:

--
562bd65a368f6194405c4ae6900e3b4388a5ec03 by Wenyi Zhao <reyizero@gmail.com>:

[MLIR][DISC] bufferize DynamicReshape and DynamicBroadcastInDim

1, add hlo-to-lhlo support for DynamicReshape and DynamicBroadcastInDim

2, add a flag `convert-to-lmhlo-only` to seperate following two case:
   - hlo-to-lhlo only. Simply lowers all mhlo ops to their lmhlo
     counterparts, do not apply any optimization (e.g. elide any
     buffer copy). Buffer optimization is not easy in dynamic
     shape world especially when involving control flow, thus we
     leave this to another dedicated pass.

   - hlo-to-lhlo-or-memref-directly. Lowers some metadata-only mhlo
     ops (e.g. reshape) to memref dialect directly and Lowers others
     to their lmhlo counterparts.

PiperOrigin-RevId: 377603395
2021-06-04 15:36:03 -07:00
A. Unique TensorFlower 8b3a75ea25 Integrate LLVM at llvm/llvm-project@b109172d99
Updates LLVM usage to match
[b109172d993e](https://github.com/llvm/llvm-project/commit/b109172d993e)

PiperOrigin-RevId: 377549160
2021-06-04 11:11:34 -07:00
A. Unique TensorFlower 9c895af2f1 Integrate LLVM at llvm/llvm-project@23a116c8c4
Updates LLVM usage to match
[23a116c8c446](https://github.com/llvm/llvm-project/commit/23a116c8c446)

PiperOrigin-RevId: 377501435
2021-06-04 06:46:18 -07:00
A. Unique TensorFlower f1f4c903df Integrate LLVM at llvm/llvm-project@fcf8827a98
Updates LLVM usage to match
[fcf8827a98be](https://github.com/llvm/llvm-project/commit/fcf8827a98be)

PiperOrigin-RevId: 377485560
2021-06-04 04:31:28 -07:00
A. Unique TensorFlower db05388a3c Integrate LLVM at llvm/llvm-project@da3ed58b97
Updates LLVM usage to match
[da3ed58b97c1](https://github.com/llvm/llvm-project/commit/da3ed58b97c1)

PiperOrigin-RevId: 377432380
2021-06-03 20:45:18 -07:00
A. Unique TensorFlower aba16adfa5 Add `mhlo.all_gather` op to MHLO dialect.
Adds import/export/verifier support as well.
Also makes `channel_handle` uniform across mhlo.all_reduce and mhlo.all-gather.

PiperOrigin-RevId: 377323468
2021-06-03 10:45:29 -07:00
Jacques Pienaar 4fc2e87a42 Add mhlo python binding generator target
This just invokes the generator backend & creates a filegroup.

PiperOrigin-RevId: 377318653
2021-06-03 10:26:30 -07:00
A. Unique TensorFlower fe42a08fc9 Use channel_handle for ChannelHandles in MHLO ops. This makes the naming of these properties consistent across these ops.
PiperOrigin-RevId: 377309518
2021-06-03 09:49:47 -07:00
A. Unique TensorFlower 063086dd78 Integrate LLVM at llvm/llvm-project@c89dff5855
Updates LLVM usage to match
[c89dff5855bb](https://github.com/llvm/llvm-project/commit/c89dff5855bb)

PiperOrigin-RevId: 377199586
2021-06-02 19:36:24 -07:00
A. Unique TensorFlower 4620410f18 Integrate LLVM at llvm/llvm-project@b25546a4b4
Updates LLVM usage to match
[b25546a4b406](https://github.com/llvm/llvm-project/commit/b25546a4b406)

PiperOrigin-RevId: 377077163
2021-06-02 09:32:59 -07:00
A. Unique TensorFlower 75a1c450ea [MLIR][KernelGen] Fix Windows build failure
Fix usage of default constructor. Instead, always use the parameterized
constructor and make the maximum supported rank explicit.

PiperOrigin-RevId: 377037155
2021-06-02 05:34:44 -07:00
A. Unique TensorFlower 557e56362e [MLIR][KernelGen] Simplify rank specialization tests with smaller target rank
For the tests rank specialize only up to rank 3. The remaining cases for higher
ranks are analogous.

PiperOrigin-RevId: 377024370
2021-06-02 03:48:07 -07:00
wyzhao 968d4b8709 PR #49598: [MLIR][DISC] legalize tensor_load inserted during hlo-to-lhlo conversion
Imported from GitHub PR https://github.com/tensorflow/tensorflow/pull/49598

This PR implements logic for lowering memref.tensor_load ops that are
inserted during `mhlo-legalize-to-lmhlo`
Copybara import of the project:

--
80eb377af4e02182e1aecc943a41ca5d7d1c2100 by Wenyi Zhao <reyizero@gmail.com>:

[MLIR][DISC] legalize tensor_load inserted during hlo-to-lhlo conversion

This PR implements logic for lowering memref.tensor_load ops that are
inserted during `mhlo-legalize-to-lmhlo`.

--
ac452fe3dcd591211cd5c59be9189fe2f7153b41 by Wenyi Zhao <reyizero@gmail.com>:

minor fix

--
6b36017f8632a06adbc3e05a62975fa641d0260f by Wenyi Zhao <reyizero@gmail.com>:

minor refine

--
846005cc76d0033112e47825c2e9a97790b6925f by Wenyi Zhao <reyizero@gmail.com>:

minor fix

--
f6a4becaa287d5ca323b2d152a4d0ae053730fd9 by Wenyi Zhao <reyizero@gmail.com>:

fix

--
5555749f60f7fce8f57962860ef65efccf0362ba by Wenyi Zhao <reyizero@gmail.com>:

fix

--
8873b9b6d9315c1199ca9f7c133ecf377ecd2fa6 by Wenyi Zhao <reyizero@gmail.com>:

fix

PiperOrigin-RevId: 376942547
2021-06-01 16:27:56 -07:00
A. Unique TensorFlower 5baf6e7709 Integrate LLVM at llvm/llvm-project@97d234935f
Updates LLVM usage to match
[97d234935f15](https://github.com/llvm/llvm-project/commit/97d234935f15)

PiperOrigin-RevId: 376840022
2021-06-01 08:38:32 -07:00
A. Unique TensorFlower d1828625ab [MLIR][KernelGen] Make maximum supported rank in rank specialization configurable
The maximum supported target rank of 5 is sufficient for all operations but
`select`. Make the maximum target rank configurable in the rank specialization.
This reduces the number of generated kernels for operations that don't require
it.

PiperOrigin-RevId: 376822496
2021-06-01 06:54:31 -07:00
A. Unique TensorFlower c7c245eaf1 [MLIR][KernelGen] Add MLIR-generated Xlogy kernel
Add the first MLIR-generated kernel that relies on an in-TF lowering. Fusion for
this kernel relies on the generalized rank specialization for operation groups.

PiperOrigin-RevId: 376805435
2021-06-01 04:48:18 -07:00
A. Unique TensorFlower 6a570502b6 Integrate LLVM at llvm/llvm-project@f000c4cfb6
Updates LLVM usage to match
[f000c4cfb66c](https://github.com/llvm/llvm-project/commit/f000c4cfb66c)

PiperOrigin-RevId: 376798033
2021-06-01 03:51:22 -07:00
A. Unique TensorFlower f16e5a3a67 [MLIR][HLO] Use canonicalization patterns in broadcast propagation pass
Replace local canonicalization patterns with those from upstream.

PiperOrigin-RevId: 376794178
2021-06-01 03:14:31 -07:00
A. Unique TensorFlower adff130b12 Integrate LLVM at llvm/llvm-project@41d7909368
Updates LLVM usage to match
[41d7909368be](https://github.com/llvm/llvm-project/commit/41d7909368be)

PiperOrigin-RevId: 376785495
2021-06-01 02:10:18 -07:00
A. Unique TensorFlower 31536431e0 [MLIR][HLO] Eliminate duplicate broadcastable constraints
PiperOrigin-RevId: 376718433
2021-05-31 13:50:23 -07:00
A. Unique TensorFlower 0f341012c6 [MLIR][HLO] Eliminate duplicate broadcastable constraints
PiperOrigin-RevId: 376715240
2021-05-31 13:08:02 -07:00
A. Unique TensorFlower 511a1db4f3 [MLIR][HLO] Use canonicalization patterns in broadcast propagation pass
Replace local canonicalization patterns with those from upstream.

PiperOrigin-RevId: 376708719
2021-05-31 12:01:26 -07:00
A. Unique TensorFlower 5f5db13715 [MLIR][HLO] Use canonicalization patterns in broadcast propagation pass
Replace local canonicalization patterns with those from upstream.

PiperOrigin-RevId: 376707588
2021-05-31 11:44:50 -07:00
A. Unique TensorFlower 1f786eb934 Integrate LLVM at llvm/llvm-project@c36ff6424f
Updates LLVM usage to match
[c36ff6424f24](https://github.com/llvm/llvm-project/commit/c36ff6424f24)

PiperOrigin-RevId: 376687137
2021-05-31 08:16:39 -07:00
A. Unique TensorFlower cc1b22e618 [HLO][Linalg] Support scalar broadcasts in point-wise converter
This is needed for operations that support this limited form of broadcasting,
namely `mhlo.select`.

PiperOrigin-RevId: 376655844
2021-05-31 03:50:23 -07:00
A. Unique TensorFlower 832f39b871 Integrate LLVM at llvm/llvm-project@577fea4e1a
Updates LLVM usage to match
[577fea4e1a13](https://github.com/llvm/llvm-project/commit/577fea4e1a13)

PiperOrigin-RevId: 376640971
2021-05-31 01:54:46 -07:00
A. Unique TensorFlower 8110683c0b Integrate LLVM at llvm/llvm-project@9968896cd6
Updates LLVM usage to match
[9968896cd62a](https://github.com/llvm/llvm-project/commit/9968896cd62a)

PiperOrigin-RevId: 376382711
2021-05-28 12:09:55 -07:00
A. Unique TensorFlower e422ce5a9a Integrate LLVM at llvm/llvm-project@8086f9d87e
Updates LLVM usage to match
[8086f9d87ee8](https://github.com/llvm/llvm-project/commit/8086f9d87ee8)

PiperOrigin-RevId: 376289371
2021-05-27 18:20:22 -07:00
Hanhan Wang 402b74ed7f Fix type bug in mhlo.dynamic-update-slice lowering.
The operand type can be f32. We should not use operand type to do clamp
operations.

PiperOrigin-RevId: 376286524
2021-05-27 17:53:49 -07:00
A. Unique TensorFlower b5f444232f Integrate LLVM at llvm/llvm-project@d47dd11071
Updates LLVM usage to match
[d47dd1107132](https://github.com/llvm/llvm-project/commit/d47dd1107132)

PiperOrigin-RevId: 376269354
2021-05-27 16:06:54 -07:00
Benjamin Kramer 6ea8f1ddf8 Integrate LLVM at llvm/llvm-project@eba69b59d1
Updates LLVM usage to match
[eba69b59d1a3](https://github.com/llvm/llvm-project/commit/eba69b59d1a3)

PiperOrigin-RevId: 376175310
2021-05-27 08:37:41 -07:00
Benjamin Kramer e319f87857 Integrate LLVM at llvm/llvm-project@c467585682
Updates LLVM usage to match
[c467585682dc](https://github.com/llvm/llvm-project/commit/c467585682dc)

PiperOrigin-RevId: 376155243
2021-05-27 06:25:17 -07:00
Benjamin Kramer 5deed55dc3 Integrate LLVM at llvm/llvm-project@d82f2a123f
Updates LLVM usage to match
[d82f2a123f9c](https://github.com/llvm/llvm-project/commit/d82f2a123f9c)

PiperOrigin-RevId: 376140722
2021-05-27 04:25:36 -07:00
A. Unique TensorFlower b32f885ad7 [MLIR][KernelGen] Enable cluster rank specialization
Replace the previously used `TransformUnrankedHloPass` which rank-specializes
only one operation at a time. The new generalized rank specialization clusters
compatible operations and rank-specializes them collectively.

PiperOrigin-RevId: 376127752
2021-05-27 02:44:31 -07:00
Adrian Kuegel a4fa6afa07 [mlir][hlo] Avoid dyn_cast_or_null when called with getDefiningOp result (NFC)
PiperOrigin-RevId: 376110457
2021-05-27 00:20:42 -07:00
A. Unique TensorFlower d939a156d8 Integrate LLVM at llvm/llvm-project@5852582532
Updates LLVM usage to match
[5852582532b3](https://github.com/llvm/llvm-project/commit/5852582532b3)

PiperOrigin-RevId: 376059531
2021-05-26 17:01:04 -07:00
Hanhan Wang 28c411606f Add support for lowering mhlo.dynamic-update-slice ops to Linalg and std ops.
PiperOrigin-RevId: 376042810
2021-05-26 15:31:05 -07:00
A. Unique TensorFlower 1c7415ba0b Integrate LLVM at llvm/llvm-project@c5c1ec7945
Updates LLVM usage to match
[c5c1ec7945ff](https://github.com/llvm/llvm-project/commit/c5c1ec7945ff)

PiperOrigin-RevId: 376014620
2021-05-26 13:21:07 -07:00
Mehdi Amini af01c08ce6 Allow tuple as results of mhlo.custom_call op (NFC)
PiperOrigin-RevId: 376009518
2021-05-26 12:56:10 -07:00
Robert Suderman 26a0053d7d Remove linalg.indexed_generic from mhlo lowerings to linalg
IndexedGeneric is going away. Transition to using linalg.Index instead.

PiperOrigin-RevId: 376002501
2021-05-26 12:24:23 -07:00
Benjamin Kramer ca09dabf1a Integrate LLVM at llvm/llvm-project@ea91a8cbab
Updates LLVM usage to match
[ea91a8cbab93](https://github.com/llvm/llvm-project/commit/ea91a8cbab93)

PiperOrigin-RevId: 375987155
2021-05-26 11:15:30 -07:00
A. Unique TensorFlower 4ebcebf31c [MLIR][HLO] Exploit scalar properties in rank specialization lowering
Take advantage of the fact that scalars are already ranked and that they are
neutral elements to broadcasting. Do not reshape scalars, do not consider them
for broadcasting, and materialize ranked operations on scalars accordingly.

PiperOrigin-RevId: 375968371
2021-05-26 09:59:13 -07:00
Benjamin Kramer 2f8f3d692c Integrate LLVM at llvm/llvm-project@01120fe5b3
Updates LLVM usage to match
[01120fe5b398](https://github.com/llvm/llvm-project/commit/01120fe5b398)

PiperOrigin-RevId: 375937201
2021-05-26 07:23:01 -07:00
Benjamin Kramer edf5ec8084 Integrate LLVM at llvm/llvm-project@cb65419b1a
Updates LLVM usage to match
[cb65419b1ac0](https://github.com/llvm/llvm-project/commit/cb65419b1ac0)

PiperOrigin-RevId: 375915516
2021-05-26 04:47:24 -07:00
A. Unique TensorFlower 4b3e9373e6 Integrate LLVM at llvm/llvm-project@bc56620b8b
Updates LLVM usage to match
[bc56620b8b76](https://github.com/llvm/llvm-project/commit/bc56620b8b76)

PiperOrigin-RevId: 375829158
2021-05-25 17:17:28 -07:00
A. Unique TensorFlower 1439ec7cae Integrate LLVM at llvm/llvm-project@0bbb502daa
Updates LLVM usage to match
[0bbb502daa90](https://github.com/llvm/llvm-project/commit/0bbb502daa90)

PiperOrigin-RevId: 375778977
2021-05-25 13:13:08 -07:00
A. Unique TensorFlower cb46298a07 [MLIR][HLO] Support all smaller ranks in rank specialization cases
Rank specialization cases can be applied to all argument tensors of smaller
ranks than the expected maximum rank. This is crucial if all operands are
effectively scalars and the maximum reduced rank is 0.

PiperOrigin-RevId: 375712020
2021-05-25 08:38:53 -07:00
Benjamin Kramer c5af02fd8d Integrate LLVM at llvm/llvm-project@f3c577ed38
Updates LLVM usage to match
[f3c577ed38e5](https://github.com/llvm/llvm-project/commit/f3c577ed38e5)

PiperOrigin-RevId: 375703642
2021-05-25 07:53:35 -07:00