Commit Graph

1046 Commits

Author SHA1 Message Date
A. Unique TensorFlower 75a1c450ea [MLIR][KernelGen] Fix Windows build failure
Fix usage of default constructor. Instead, always use the parameterized
constructor and make the maximum supported rank explicit.

PiperOrigin-RevId: 377037155
2021-06-02 05:34:44 -07:00
A. Unique TensorFlower 557e56362e [MLIR][KernelGen] Simplify rank specialization tests with smaller target rank
For the tests rank specialize only up to rank 3. The remaining cases for higher
ranks are analogous.

PiperOrigin-RevId: 377024370
2021-06-02 03:48:07 -07:00
wyzhao 968d4b8709 PR #49598: [MLIR][DISC] legalize tensor_load inserted during hlo-to-lhlo conversion
Imported from GitHub PR https://github.com/tensorflow/tensorflow/pull/49598

This PR implements logic for lowering memref.tensor_load ops that are
inserted during `mhlo-legalize-to-lmhlo`
Copybara import of the project:

--
80eb377af4e02182e1aecc943a41ca5d7d1c2100 by Wenyi Zhao <reyizero@gmail.com>:

[MLIR][DISC] legalize tensor_load inserted during hlo-to-lhlo conversion

This PR implements logic for lowering memref.tensor_load ops that are
inserted during `mhlo-legalize-to-lmhlo`.

--
ac452fe3dcd591211cd5c59be9189fe2f7153b41 by Wenyi Zhao <reyizero@gmail.com>:

minor fix

--
6b36017f8632a06adbc3e05a62975fa641d0260f by Wenyi Zhao <reyizero@gmail.com>:

minor refine

--
846005cc76d0033112e47825c2e9a97790b6925f by Wenyi Zhao <reyizero@gmail.com>:

minor fix

--
f6a4becaa287d5ca323b2d152a4d0ae053730fd9 by Wenyi Zhao <reyizero@gmail.com>:

fix

--
5555749f60f7fce8f57962860ef65efccf0362ba by Wenyi Zhao <reyizero@gmail.com>:

fix

--
8873b9b6d9315c1199ca9f7c133ecf377ecd2fa6 by Wenyi Zhao <reyizero@gmail.com>:

fix

PiperOrigin-RevId: 376942547
2021-06-01 16:27:56 -07:00
A. Unique TensorFlower 5baf6e7709 Integrate LLVM at llvm/llvm-project@97d234935f
Updates LLVM usage to match
[97d234935f15](https://github.com/llvm/llvm-project/commit/97d234935f15)

PiperOrigin-RevId: 376840022
2021-06-01 08:38:32 -07:00
A. Unique TensorFlower d1828625ab [MLIR][KernelGen] Make maximum supported rank in rank specialization configurable
The maximum supported target rank of 5 is sufficient for all operations but
`select`. Make the maximum target rank configurable in the rank specialization.
This reduces the number of generated kernels for operations that don't require
it.

PiperOrigin-RevId: 376822496
2021-06-01 06:54:31 -07:00
A. Unique TensorFlower c7c245eaf1 [MLIR][KernelGen] Add MLIR-generated Xlogy kernel
Add the first MLIR-generated kernel that relies on an in-TF lowering. Fusion for
this kernel relies on the generalized rank specialization for operation groups.

PiperOrigin-RevId: 376805435
2021-06-01 04:48:18 -07:00
A. Unique TensorFlower 6a570502b6 Integrate LLVM at llvm/llvm-project@f000c4cfb6
Updates LLVM usage to match
[f000c4cfb66c](https://github.com/llvm/llvm-project/commit/f000c4cfb66c)

PiperOrigin-RevId: 376798033
2021-06-01 03:51:22 -07:00
A. Unique TensorFlower f16e5a3a67 [MLIR][HLO] Use canonicalization patterns in broadcast propagation pass
Replace local canonicalization patterns with those from upstream.

PiperOrigin-RevId: 376794178
2021-06-01 03:14:31 -07:00
A. Unique TensorFlower adff130b12 Integrate LLVM at llvm/llvm-project@41d7909368
Updates LLVM usage to match
[41d7909368be](https://github.com/llvm/llvm-project/commit/41d7909368be)

PiperOrigin-RevId: 376785495
2021-06-01 02:10:18 -07:00
A. Unique TensorFlower 31536431e0 [MLIR][HLO] Eliminate duplicate broadcastable constraints
PiperOrigin-RevId: 376718433
2021-05-31 13:50:23 -07:00
A. Unique TensorFlower 0f341012c6 [MLIR][HLO] Eliminate duplicate broadcastable constraints
PiperOrigin-RevId: 376715240
2021-05-31 13:08:02 -07:00
A. Unique TensorFlower 511a1db4f3 [MLIR][HLO] Use canonicalization patterns in broadcast propagation pass
Replace local canonicalization patterns with those from upstream.

PiperOrigin-RevId: 376708719
2021-05-31 12:01:26 -07:00
A. Unique TensorFlower 5f5db13715 [MLIR][HLO] Use canonicalization patterns in broadcast propagation pass
Replace local canonicalization patterns with those from upstream.

PiperOrigin-RevId: 376707588
2021-05-31 11:44:50 -07:00
A. Unique TensorFlower 1f786eb934 Integrate LLVM at llvm/llvm-project@c36ff6424f
Updates LLVM usage to match
[c36ff6424f24](https://github.com/llvm/llvm-project/commit/c36ff6424f24)

PiperOrigin-RevId: 376687137
2021-05-31 08:16:39 -07:00
A. Unique TensorFlower cc1b22e618 [HLO][Linalg] Support scalar broadcasts in point-wise converter
This is needed for operations that support this limited form of broadcasting,
namely `mhlo.select`.

PiperOrigin-RevId: 376655844
2021-05-31 03:50:23 -07:00
A. Unique TensorFlower 832f39b871 Integrate LLVM at llvm/llvm-project@577fea4e1a
Updates LLVM usage to match
[577fea4e1a13](https://github.com/llvm/llvm-project/commit/577fea4e1a13)

PiperOrigin-RevId: 376640971
2021-05-31 01:54:46 -07:00
A. Unique TensorFlower 8110683c0b Integrate LLVM at llvm/llvm-project@9968896cd6
Updates LLVM usage to match
[9968896cd62a](https://github.com/llvm/llvm-project/commit/9968896cd62a)

PiperOrigin-RevId: 376382711
2021-05-28 12:09:55 -07:00
A. Unique TensorFlower e422ce5a9a Integrate LLVM at llvm/llvm-project@8086f9d87e
Updates LLVM usage to match
[8086f9d87ee8](https://github.com/llvm/llvm-project/commit/8086f9d87ee8)

PiperOrigin-RevId: 376289371
2021-05-27 18:20:22 -07:00
Hanhan Wang 402b74ed7f Fix type bug in mhlo.dynamic-update-slice lowering.
The operand type can be f32. We should not use operand type to do clamp
operations.

PiperOrigin-RevId: 376286524
2021-05-27 17:53:49 -07:00
A. Unique TensorFlower b5f444232f Integrate LLVM at llvm/llvm-project@d47dd11071
Updates LLVM usage to match
[d47dd1107132](https://github.com/llvm/llvm-project/commit/d47dd1107132)

PiperOrigin-RevId: 376269354
2021-05-27 16:06:54 -07:00
Benjamin Kramer 6ea8f1ddf8 Integrate LLVM at llvm/llvm-project@eba69b59d1
Updates LLVM usage to match
[eba69b59d1a3](https://github.com/llvm/llvm-project/commit/eba69b59d1a3)

PiperOrigin-RevId: 376175310
2021-05-27 08:37:41 -07:00
Benjamin Kramer e319f87857 Integrate LLVM at llvm/llvm-project@c467585682
Updates LLVM usage to match
[c467585682dc](https://github.com/llvm/llvm-project/commit/c467585682dc)

PiperOrigin-RevId: 376155243
2021-05-27 06:25:17 -07:00
Benjamin Kramer 5deed55dc3 Integrate LLVM at llvm/llvm-project@d82f2a123f
Updates LLVM usage to match
[d82f2a123f9c](https://github.com/llvm/llvm-project/commit/d82f2a123f9c)

PiperOrigin-RevId: 376140722
2021-05-27 04:25:36 -07:00
A. Unique TensorFlower b32f885ad7 [MLIR][KernelGen] Enable cluster rank specialization
Replace the previously used `TransformUnrankedHloPass` which rank-specializes
only one operation at a time. The new generalized rank specialization clusters
compatible operations and rank-specializes them collectively.

PiperOrigin-RevId: 376127752
2021-05-27 02:44:31 -07:00
Adrian Kuegel a4fa6afa07 [mlir][hlo] Avoid dyn_cast_or_null when called with getDefiningOp result (NFC)
PiperOrigin-RevId: 376110457
2021-05-27 00:20:42 -07:00
A. Unique TensorFlower d939a156d8 Integrate LLVM at llvm/llvm-project@5852582532
Updates LLVM usage to match
[5852582532b3](https://github.com/llvm/llvm-project/commit/5852582532b3)

PiperOrigin-RevId: 376059531
2021-05-26 17:01:04 -07:00
Hanhan Wang 28c411606f Add support for lowering mhlo.dynamic-update-slice ops to Linalg and std ops.
PiperOrigin-RevId: 376042810
2021-05-26 15:31:05 -07:00
A. Unique TensorFlower 1c7415ba0b Integrate LLVM at llvm/llvm-project@c5c1ec7945
Updates LLVM usage to match
[c5c1ec7945ff](https://github.com/llvm/llvm-project/commit/c5c1ec7945ff)

PiperOrigin-RevId: 376014620
2021-05-26 13:21:07 -07:00
Mehdi Amini af01c08ce6 Allow tuple as results of mhlo.custom_call op (NFC)
PiperOrigin-RevId: 376009518
2021-05-26 12:56:10 -07:00
Robert Suderman 26a0053d7d Remove linalg.indexed_generic from mhlo lowerings to linalg
IndexedGeneric is going away. Transition to using linalg.Index instead.

PiperOrigin-RevId: 376002501
2021-05-26 12:24:23 -07:00
Benjamin Kramer ca09dabf1a Integrate LLVM at llvm/llvm-project@ea91a8cbab
Updates LLVM usage to match
[ea91a8cbab93](https://github.com/llvm/llvm-project/commit/ea91a8cbab93)

PiperOrigin-RevId: 375987155
2021-05-26 11:15:30 -07:00
A. Unique TensorFlower 4ebcebf31c [MLIR][HLO] Exploit scalar properties in rank specialization lowering
Take advantage of the fact that scalars are already ranked and that they are
neutral elements to broadcasting. Do not reshape scalars, do not consider them
for broadcasting, and materialize ranked operations on scalars accordingly.

PiperOrigin-RevId: 375968371
2021-05-26 09:59:13 -07:00
Benjamin Kramer 2f8f3d692c Integrate LLVM at llvm/llvm-project@01120fe5b3
Updates LLVM usage to match
[01120fe5b398](https://github.com/llvm/llvm-project/commit/01120fe5b398)

PiperOrigin-RevId: 375937201
2021-05-26 07:23:01 -07:00
Benjamin Kramer edf5ec8084 Integrate LLVM at llvm/llvm-project@cb65419b1a
Updates LLVM usage to match
[cb65419b1ac0](https://github.com/llvm/llvm-project/commit/cb65419b1ac0)

PiperOrigin-RevId: 375915516
2021-05-26 04:47:24 -07:00
A. Unique TensorFlower 4b3e9373e6 Integrate LLVM at llvm/llvm-project@bc56620b8b
Updates LLVM usage to match
[bc56620b8b76](https://github.com/llvm/llvm-project/commit/bc56620b8b76)

PiperOrigin-RevId: 375829158
2021-05-25 17:17:28 -07:00
A. Unique TensorFlower 1439ec7cae Integrate LLVM at llvm/llvm-project@0bbb502daa
Updates LLVM usage to match
[0bbb502daa90](https://github.com/llvm/llvm-project/commit/0bbb502daa90)

PiperOrigin-RevId: 375778977
2021-05-25 13:13:08 -07:00
A. Unique TensorFlower cb46298a07 [MLIR][HLO] Support all smaller ranks in rank specialization cases
Rank specialization cases can be applied to all argument tensors of smaller
ranks than the expected maximum rank. This is crucial if all operands are
effectively scalars and the maximum reduced rank is 0.

PiperOrigin-RevId: 375712020
2021-05-25 08:38:53 -07:00
Benjamin Kramer c5af02fd8d Integrate LLVM at llvm/llvm-project@f3c577ed38
Updates LLVM usage to match
[f3c577ed38e5](https://github.com/llvm/llvm-project/commit/f3c577ed38e5)

PiperOrigin-RevId: 375703642
2021-05-25 07:53:35 -07:00
Adrian Kuegel a847109ac7 Support complex types when converting HLO multiply op.
We can lower it to the MulOp in the complex dialect.

PiperOrigin-RevId: 375675079
2021-05-25 04:35:34 -07:00
Adrian Kuegel 5816920258 Support complex types when converting HLO divide op.
We can lower it to the DivOp in the complex dialect.
Also add tests to hlo-legalize-to-linalg.mlir for CompareOp lowering of complex
types. These were forgotten in a previous commit.

PiperOrigin-RevId: 375669125
2021-05-25 03:43:46 -07:00
Benjamin Kramer 8e28008e38 Integrate LLVM at llvm/llvm-project@6359842bc0
Updates LLVM usage to match
[6359842bc088](https://github.com/llvm/llvm-project/commit/6359842bc088)

PiperOrigin-RevId: 375667591
2021-05-25 03:30:27 -07:00
Adrian Kuegel 758ae7da6b Support complex types when converting HLO compare op (EQ/NE).
We can lower it to the EqualOp / NotEqualOp in the complex dialect.

PiperOrigin-RevId: 375655092
2021-05-25 01:54:27 -07:00
A. Unique TensorFlower 5504f82f11 Integrate LLVM at llvm/llvm-project@c2f819af73
Updates LLVM usage to match
[c2f819af73c5](https://github.com/llvm/llvm-project/commit/c2f819af73c5)

PiperOrigin-RevId: 375598055
2021-05-24 17:52:41 -07:00
A. Unique TensorFlower af6092ef13 Integrate LLVM at llvm/llvm-project@d65c32fb41
Updates LLVM usage to match
[d65c32fb41b0](https://github.com/llvm/llvm-project/commit/d65c32fb41b0)

PiperOrigin-RevId: 375537865
2021-05-24 12:57:30 -07:00
wyzhao b93e54d8a4 PR #49454: [MLIR][DISC] Upgrade to use the new `reifyReturnTypeShapes` interface.
Imported from GitHub PR https://github.com/tensorflow/tensorflow/pull/49454

The new interface is more safe to be used during dialect conversion
(e.g. converting from tensor world to buffer world).
Copybara import of the project:

--
a6968072d59bec3c3bbaef0121d297e807c37c91 by Wenyi Zhao <reyizero@gmail.com>:

[MLIR][DISC] Upgrade to use the new `reifyReturnTypeShapes` interface.

The new interface is more safe to be used during dialect conversion
(e.g. converting from tensor world to buffer world).

--
55e7c6b7f2f99b99e226645a57e2433fae3e90ed by Wenyi Zhao <reyizero@gmail.com>:

minor fix

PiperOrigin-RevId: 375500273
2021-05-24 10:11:55 -07:00
Stella Laurenzo 28c4112f35 Downgrade some emitErrors in patterns to notifyMatchFailure.
PiperOrigin-RevId: 375160543
2021-05-21 14:09:23 -07:00
Hanhan Wang 1ba4c714c9 Add support for lowering mhlo.scatter ops to Linalg.
This only works for updating tensors, not add/min/max computations. It requires
the index depth to be 1 because of the limitation in Linalg. We can not compare
multiple indices without packing indices.

PiperOrigin-RevId: 375137721
2021-05-21 12:17:14 -07:00
A. Unique TensorFlower f5dc73ab60 Integrate LLVM at llvm/llvm-project@eb6429d0fb
Updates LLVM usage to match
[eb6429d0fb94](https://github.com/llvm/llvm-project/commit/eb6429d0fb94)

PiperOrigin-RevId: 375071389
2021-05-21 06:31:13 -07:00
A. Unique TensorFlower 1f996ab796 Integrate LLVM at llvm/llvm-project@b9076d119a
Updates LLVM usage to match
[b9076d119a5b](https://github.com/llvm/llvm-project/commit/b9076d119a5b)

PiperOrigin-RevId: 375055490
2021-05-21 04:09:26 -07:00
A. Unique TensorFlower 97e6103933 [MLIR][HLO] Reshape to scalars in rank specialization
Scalars were incorrectly casted to scalar tensors when they have to be reshaped.

PiperOrigin-RevId: 375049088
2021-05-21 03:12:16 -07:00